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ABSTRACT 
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hultisource  Messages,  Even  Radar)  is  an  experimental  program 
created  as  part  cf  en  investigation  into  methods  of 
correlating  information  in  the  naval  environment.  This 
thesis  is  an  exploration  into  the  application  cf 
artificial  intelligence  to  the  tactical  situation 
assessment  problem  and  into  various  evaluation 
methodologies  for  STAMMER2.  Included  is  an  overview  of 
one  c:  these  experiments,  using  the  facilities  cf  the 
Naval  Postgraduate  School  Command,  Control  and 
Communications  Laboratory  and  tne  Naval  Ccean  Systems 
Center,  San  Diego,  California. 
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I.   INTRCEUCTIC:. 


A.   IACKC-RCUND 

STAMM2R2   (System   for    Tactical    Assessment    cf 
Multisource   Messages,    Even  Radar)   is   an  experimental 


computer  program  created  as  part 


an 


investigation 


into  new  methods  cf  correlating  information  available  in 
a  naval  environment  in  response  to  trie  tactical  situation 
assessment  (TSA)  problem.  It  was  written  by  the  Tactical 
Commend  and  Control  Division  of  the  Naval  Ocean  Systems 
Center,  San  Diego,  California.  STAKMER2  has  served  as  a 
testbed  for  explorations  of  applications  of  rule-based 
artificial  intelligence  inference  systems,  with  a  general 
end  flexible  approach  to  the  ran.-;e  of  acceptable  inputs 
tc  the  system. 

In  practical  application,  information  in  a  oasic 
forrat  is  received  from  a  variety  of  sensors.  This 
information  is  processed  within  a  network  of 
interconnected  rules  or  conditional  statements  which 
may  be  satisfied  by  this  inf crmaticn ,  even 
responding  to  information  in  cases  involving  varying 
degrees  cf  uncertainty.  STAMM2R2  is  capable  cf 
performing  deductions  based  on  this  received  information 
tc  speed  the  decision  process  in  a  rapidly  changing 
environment . 


Befcre  we  discuss  tne  specifics  cf  STAMMSR2  ar.i  a 
method  of  testing  the  utility  of  this  decision  aid, 
background  information  will  te  presented  to  set  the  staee 
for  the  why,  where,  end  how  such  a  system  was  developed. 

B.   OBJECTIVES 

There  ere  three  principal  objectives  for  this  thesis: 

1.  Identify  the  characteristics  of  an  artificial 
intelligence  prcgram,  with  an  overview  cf  the 
applications  to  a  specific  military  situation. 

2.  Propose  various  levels  of  evaluation  methodologies 
for  STAMMER2. 

3.  Present  the  results  of  an  experiment  conducted 
with  STAMMER2  using  one  of  ^ the  methodologies 
ore sen  ted  . 
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II  .       ARTIFICIAL    IMTZIUGZNCZ 

A.  DEFINITION 

Artificial  intelligence  (AI)  is  defined  as  "the  study 
of  ideas  which  enable  computers  tc  dc  the  things  that  make 
people  seer  intelligent"  rRef.  1]  end  "the  science  of  making 
machines  do  things  that  would  require  intelligence  if  dene 
by  men"  [Ref .  2]  . 

The  intricate  processes  from  which  these  "things" 
have  evolved,  such  as  learning,  growth,  or  maturation 
are  bypassed  in  the  artificial  intelligence  program  to 
create  a  facsimile  of  human  thought  and,  subsequently, 
decision-making.  rut  the  bypassing  is,  in  reality,  a  very 
careful  compilation  of  the  elements  which  are,  or  seem  to 
be,  parts  cf  the  human  thought  patterns  and  strategies 
leading  to  decisions.  That  is,  human  thought  is  analyzed 
and  dissected  into  discrete  elements  which  may  be 
recombinei  artificially  to  mimic  human  intelligence. 
Alternatively,  differing  patterns  or  methods  are 
explored  to  uncover  new  routes  which  could,  in  fact,  lead  to 
similar  results. 

B.  DISCUSSION 

To  imagine  tne  thought  process  as  a  branched  tree, 
with  decisions   made   at    each   juncture,    cr   node, 
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is    a  s Imp li fleet  ion   which  does  not  reflect  the  complexity 
of   the   problem    of   describing   human   thought.    The 

cctuel   representation  must   consider  many   interconnected 

paths,   with  multidirectional   flow   through   those   paths. 

A   control   mechanism   must    also    he   considered   which 

carries   a   thought   through  to   conclusion  and   minimizes 

retraced   cr  ncn-prcductive  rcutes  . 

A  goal  of  artificial  intelligence  inquiry  is  to 
duplicate  this  complex  structure  in  a  computer  program.  A 
difficulty  lies  in  the  multi-dimensional  nature  of  the 
thought  process  which  requires  equally  complex 
investigation.  By  examining  the  problem  from  the  viewpoint 
of  discrete  elements,  it  may  be  possible  to  find  the 
combination  which  accurately  reflects  a  pattern  which  net 
only  duplicates  what  happens  in  the  case  of  a  specific 
problem,  but  also  a  general  case  which  may  be  applicable  to 
ether  prcolems . 

If  STAMMER2  can  duplicate  the  reasoning  process  of  a 
decision  maker,  with  a  aata  base  and  memory  consistent  with 
the  situation  presented,  then  it  is  possible  to  use  this 
artificial  intelligence  program  as  an  "intelligent 
assistant"  in  the  tactical  situation  assessment  problem 
which  will  be  discussed  in  the  next  section. 

The  computer  is  the  "ideal  experimental  animal." 
Infinitely  patient  and  easy  to  care  for,  the  computer 
requires   structured   instruction,   composed   of   discrete 
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elements,  which  it  combines  as  programmed.  The  combinat  icr. 
end  recombination,  done  at  speeds  sufficient  to  expose 
unproductive  rains  rapidly,  can  provide  a  practical  test  of 
the  precision  with  whicn  the  experimenter  defines  his 
thought  model  [Ref .  3]  .  Additions  or  deletions  which 
the  experimenter  ma/  hypotnesize  as  crucial  (or  useless)  to 
the  final  result  of  his  pattern  may  he  evaluated  with 
relative  ease.  Flaws  or  conceptual  mistakes  which  may 
exist  are  likely  tc  be  detected  in  the  computer  because 
the  computer  generally  will  not  accept  ambiguity  of 
instruction  and  may  simply  cease  execution  of  the 
program.  However,  a  program  designed  to  answer  a  certain 
sort  of  question  does  not  guarantee  a  correct  answer 
[Ref.  4]  . 

The  applicability  of  computers  and  programming  is  not 
the  emphasis  of  artificial  intelligence.  However,  the 
unambiguous  nature  of  the  program  is  mentioned  to  reinforce 
the  transparency"  of  the  actual  machine  and  focus  on  the 
precision  cf  understanding  human  thought. 

Using  artificial  intelligence  techniques,  research  has 
been  done  to  explore  vision,  problem  solving,  language,  and 
neurosis  [Ref.  5].  Some  of  these  models  have  been 
successful  in  predicting  behavior  under  specified  conditions 
while  others  have  been  at  least  of  metaphorical  value  in 
helping  tc  understand  the  possible  Techanism  cf 
intelligence.  By   representing   thought   and  knowledge,   in 
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general,  in  a  useful  end  flexible  manner  it  is  also  possible 
tc  arrange  an  interaction  between  a  human  and  tne  machine  in 
a  task  oriented  environment. 

Knowledge  is  defined  as  a  collection  of  facts,  the 
state  cf  knowing,  or  all  that  has  been  perceived  by  the 
human  mind  [Ref .  6] .  Knowledge  may  be  in  human  or  machine 
remcry  cr  in  a  data  base.  It  is,  however,  also  in  the 
actual  procedures  that  operate  on  or  by  reference  to  the 
data.  Tc  store  disjointed  facts  in  computer  memory  cr  in 
a  date  base  from  which  the  labelled  information  units  may 
be  retrieved  is  not  the  goal  of  artificial 
intelligence.  The  identification  cf  the  desired  fact  (cr  a 
location  code  by  which  that  fact  nay  be  retrieved)  is  a 
problem  of  database  management.  Tc  discern  which  fact  is 
reauired  is,  however,  the  end  of  a  series  of  steps.  In 
tne  problem  solving  case,  the  solution  process  is  the 
strategy  of  gaining  information  after  a  forward  or 
backward  looking  reasoning  method"  [Ref  7]  . 

Analogies  to  other  human  processes  exist,  for  example, 
radar  provides  target  information  in  a  rranner  analogous 
tc  the  eye.  Data,  such  as  target  presence,  range,  and 
bearing  may  be  useful  to  the  operator  as  well  as  in  a  form 
which  can  be  compared  tc  characteristics  stcred  in  computer 
memory . 

The  specific  goal  of  STAMMER2  is  to  practically  combine 
this   sensory   data   with   the  problem   solving   process  of 
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data  receipt,  corrparison  and  retrieval  using   the   knowledge 
stored  in  the  artificial  intelligence  program. 


It 


III.   TACTICAL  SITUATION  ASSESSMENT  AN 2  STAMKSR2 


A.   TACTICAL  SITUATION  ASSESSMENT 

In  the  naval  context,  the  ability  of  a  commander  tc 
accurately  assess  his  military  situation  requires   decisions 

about  his  environment .  While  all  decisions  need,  not  be  rede 
en  an  inrediate  basis,  certain  cf  then,  called  tactical, 
determine  his  inrediate  course  of  action.  In  an  extreme 
example,  the  decision  tc  launcn  a  defensive  weapon  is  based 
on  s  judgment  that  such  an  action  is  required  besed  on  the 
commander's  evaluation.  A  staterent  of  the  tactical 
situation  assessment  (TSA)  prcbleT  includes  consideration 
cf  the  information  available  tc  the  decision  raker. 

In  order  tc  essess  a  military  problem,  the  evaluator 
craws  upon  experience  and  training,  after  having  considered 
the  situation  presented  tc  him.  The  assessment  of  the 
tactical  situation  is,  however,  highly  dependent  upon 
factors  associated  with  the  decision  maker  himself. 

The  experience  upon  which  a  military  decision  maker 
draws  is  unique  tc  him  alone,  although  there  is 
certainly  a  similarity  of  career  patterns  and  situations 
common  to  many  individuals.  Unique  to  that  individual  is 
the  Guality  of  his  recollection,  his  emotional  or  mental 
state,  the   perspective  of   the   individual,   and  the  length 
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cf  time  elapsed  sir.ce  those  events  frcm  which  the 
experience  was  gained. 

The  actual  decision  rraker  in  the  operations  center 
the  Combat  Information  Center  or  CIC)  cf  a  Navy  ship  is 
the  Tactical  Action  Officer  (TAO).  Luring  the  period  of 
the  Vietnam  conflict,  the  rcle  cf  the  Captain  cf  a  ship  as 
final  fighting  authority  was  altered  screwhat  from  the 
traditional  role.  The  Tactical  Acticn  Officer  concept  was 
proposed,  implemented  in  Navy  Regulations,  and  exercised  in 
combat  [Ref.  8].  The  concept  basicly  states  that  a  trusted 
subordinate,  trained  and  drilled  in  defensive  doctrine 
and  procedures,  acts  as  weapons  release  authority  in  the 
temporary  absence  cf   the   Captain. 

The  specialized  formal  training  for  TACs,  having  as 
prerequisites  proven  maturity  and  operational 
experience,  lasts  approximately  six  weeks.  The  student  is 
exposed  to  information  concerning  U.S.  weapons  systems 
and  those  of  potential  enemies.  After  this  is 
committed  to  memory,  "standard"  tactical  doctrine  is 
exercised  in  a  series  cf  increasingly  complex  scenarios. 
The  scenarios  are  presented  in  sequences  allowing 
questions  to  be  answered  based  en  information  normally 
available  to  the  TAC  (from  the  training,  experience, 
tactical   publications  f  sensors  ,  dnd.    intelligence  sources). 

The  TAO  becomes  the  focus  of  the  Combat  Information 
Center.    kll        sensor  reports   are  displayed   to  him,  all 
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ges  from  external  sources  are  given  to  him  directly   or 


in  summarized  form. 


ne 


:  a  l  stand   alone,   on   his   cvr. 


ship,  or  be  electronically  linked  q-j  radio  to  TAOs  or. 
ether  snips,  each  going  through  similar  procedures  and 
actions.  This  defines  a  sphere  of  knowledge  in  which  the 
1AC  operates,  ana  can  generally  be  delineated  by  the  range 
cf  his  sensors.  A  correlation  function  is  required  when 
the  TAG  receives  'external"  reports  from  sensors  or 
platforms  other  than  his  own — in  effect  from  a  sphere  cf 
larger  radius. 

Assumptions  involved  in  reducing  many  TAG  activities 
to  computer  form  include:  a,  the  basic,  repetitive  m^ss 
of  information  ce.r^  be  reducea  to  a  computerized  database; 
b;  standardized  procedures  for  data  retrieval  and 
correlation  can  be  sirilarly  reduced;  c)the 
sensor/intelligence  information  car.  be  presented  in  a 
standard  format;  and  d)  the  thought  process  by  which  the 
TAG  functions  may  be  artificially  reproduced. 

r.   STAMMEE2 

STAMMIR2  is  a  revised  version  cf  STAMMER,  a  System  for 
Tactical  Assessment  of  Multisource  Messages,  Even  Radar.  It 
was  created  as  part  cf  an  investigation  cf  correlation 
methodologies,  and  served  as  a  testbed  for  explorations  cf 
applications  of  rule-based  inference  systems  to  the  tactical 
situation  assessment  <TSA)  problem.  STAMMER  concentrated  en 
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the  specific  task  of  merchant  shir  discrimination  f r c r   all 
contacts  reported  by  radar  and  external  Tessa^es  [Ref .  9]   . 
STiiMMIR2    is   £l   organizer   of   information.    It 


collects   information   by  receiving  T=55:;-e« 


cr 


reports  (radar,  electronic  support  measures,  and  sccerv, 
d.zt  organizes  this  raw  data  into  graphic  displays  and 
textual  commentary  to  aid  in  tactical  situation 
assessment.  Through  tea  use  of  specified  rules,  the 
system  combines  this  inf  cr^et  ion  tc  Irav  conclusions 
about  tee  situation  in  the  vicinity  of  the  here  ship.  Incse 
combinations  are  reflected  in  both  the  STAMK5R2  display  c~d 
commentar/.  Ihe  systerr  is  available  for  the  exariaation  of 
raw  data  as  well  as  information  about  why  and  hew  the 
conclusions  were  reached. 

STAMh"EE2  deals  with  information  en  a  real-tire  basis. 
As  a  Tessa&e  is  received  from  5  sensor  or  information 
source  in  STAMMEE2-readable  format,  it  becomes  part  of 
the  data  base  cr.  which  the  rules  operate.  This  process  is 
complicated  by  the  fact  that  the  information  is  suspect,  zlz 
its  arrival  -ay  not  be  in  chrcn clerical  crder. 

Although  STA(":tfIB2  was  developed  in  response  tc 
Navy-  sponsored  investigation  of  the  TSA  problem,  the  U.S. 
^:rj      has    conducted   sinilar    investigations    in   the 


:  ieid 


of   military   intelligence   analysis 


Ref 


1  e1 

- c  -  • 


3asea  upon  a   detailed   stuay   of 

methods,   and   thought  processes  in  intelligence  production, 
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it  rray  be  possible  to  streamline  intelligence  analysis. 
In   essence,   this   :- recess   used   by   the   analyst   is   the 

rethodolc •-:;.'  from  which  the  A I  program  might  be 
developed  after  a  descriptive  mclel  is  organized  and 
exerci  sed . 

The  rranner  in  which  STAKMER2  functions  vs.;/  best  be 
described  by  analogy  to  the  medical  diagnosis  system  called 
KYCIN  [Ref.  11].  MYCIN  is  a  problem  solving  system  by 
which  a  physician,  responding  tc  queries  frc"  the  program, 
receives  assistance  in  identifying  and  treating  blcod 
bacterial  infections.  The  basis  fcr  the  expertise 
resident  in  MYCIN  is  the  knowledge  obtained  from 
interviewing  physicians  and  experts  and  stored  in  ever  3?2 
"productions".  Productions  ere  memory  structures  on  which 
the  program   operates   and  are  in  the  following  format. 

If  the  infection  type  is  primary-bacteremia , 

the  suspectea  entry  -point  is  the  gastrointestinal  tract, 
and  the  site  of  the  culture  is  one  cf  the  sterile  sites, 

then   tnere  is  evidence  that  the  organism  is  bacteroides. 

Conceptually,  inference  rules  are  very  simple.  Every 
rule  attempts  to  retrieve  information  from  remory  and,  if  it 
succeeds,  constructs  a  new  assertion  in  memory.  If  all  the 
conditions  are  matched,  then  the  rules  are  said  to  "fire". 
That  is,  ail  the  actions  which  are  defined  in  the   rule   are 
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then  carried  out.   These  actions  may  be  modifications  to  tie 


data   base,   assigning 


confidences,   and   urinting 


formatted  descriptions. 

In  a  sircle  rule  interpreter,  the  system  maintains  a 
list  cf  rules,  each  witn  conditions  which  pertain  to  a 
central  data  base,  whicn  is  constantly  being  updated.  At 
regular  intervals  the  system  attempts  tc  fire  each  rule; 
that  is,  it  checks  to  see  if  the  rule  conditions  are 
currently  satisfied  in  the  data  base.  If  ere  cf  many 
conditions  is  not  satisfied,  a  rule  ray  fail  to  fire  and,  in 
tnis  case,  any  partial  work  dene  in  satisfying  the  rule  is 
lost.  In  addition,  each  rule  is  retried  at  the  time 
interval  without  regard  tc  the  nature  cf  the  intervening 
charges  to  the  data  case,  duplicating  results  where  no 
charges  to  the  data  base  nave  occurred. 

Because  Tedical  diagnosis,  even  by  experts,  involves 
varying  decrees  of  uncertainty,  MYCIN  is  written  with  a 
so-oalled  certainty  factor  fcr  each  ccnclusicn.  The 
physicians  upon  whose  information  the  productions  were 
established  T*ere  also  askei  tc  provide  "strength 
values",  or  probabilities  cf  accuracy  and  confidence, 
for  their  assertions.  This  introduces  the  ability  to 
leal  with  uncertain  evidence  into  the  artificial 
intelligence  system. 

^'c.e  productions  used  to  reacn  a  ccnclusicn  are  part  cf 
the   conclusion   set.   MYCIN  answers  iu est  ions  ah  out  how  end 
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why  a  fact  v.as  established  or  used.  The  production  is 
recalled   (by  name  cr  number)  to  shew  an  enumeration  of  the 

facts  presented  in  the  premise  set.  G e n e r e 1  facts  can  he 
questioned  without  resorting  tc  tne  entire  diagnosis  hecause 
each  production  "stands  alone",  to  he  actively  examined  or 
questioned.  In  tnis  case  tne  steps  ta/en  to  arrive  at  that 
particular  fact  can  else  be  traced.  This  tracing  ^ay  act 
as  a  training  aid  for  the  user  in  addition  to  presenting 
alternatives  which  can  he  explored. 

The  critical  aspects  of  STA^MIH2  's  design  may  he  divided 
intc  four  parts.  These  are  ^e^cry  (the  data  case),  rule 
interpretation,  explanation,  and  graphics. 

The  rules  of  STAMMER2  consist  of  conditions,  actions  and 
confidences,  with  conditions  and  actions  in  the  same 
structure  as  assertions  with  allowances  made   for   variables 


which  are   'hound'  tc  a  rule  hy  a  binding  function  in 
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the  language  in  which  STAPMER2  is  written  [Hef.  12] .  These 
variables  which  are  applied  tc  the  rule  conditions  will  be 
evaluated  to  see  whether  tne  condition  succeeds  or  fails. 
If  it  fails,  a  "suspension"  is  created  which  corresponds  tc 
the  remainder  of  tne  rule.  As  more  information  is  added  to 
the  data  base,  those  suspensions  which  can  use  the  new  data 
are  revived  and  continue  as  before  (either  completing  or 
suspending  again).  A  suspension  contains  not  only  the 
remaining  part  of  the  rule,  but  also  the  bindings 
established   by  already  satisfied  conditions,  if  any.   Sven 
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tfher  c  condition  succeeds,  there  may  be  other  ways  for  it  tc 
be  satisfied,  so  a  suspension  is  left  behind. 

The  individual  rules   are   expressed   in   the   following 
fern. 


RULE-NAME 
(CONDITIONS 

((  <cc edition  1>  ) 
i  <  c  c  n  d  i  t  i  c  n  2  >  ; 


(  <ccndition  n>  ) ) 
ACTIONS 

1  (  <action  1>  ) 

(  <action  2>  ) 


(  <action  m>  ) 
CONI  <a  number  between  -1  and  +lv 
PRINFORK 

"This  is  a  sentence  describing  the  rule.") 

STAMMER2   deals  witii   information   en   a   real-tire 

basis.   However,   this   process  is  complicated  by  the   fact 

that   the  arrival   cf   reports  may  net  be   in   chronological 

crier,    with   later   information  superseding  or  negating 

earlier   reports.   A   'data   stream"    is   used   to   bind 

information   as   it  is  received  tc  the  rule  condition  which 

i  t  satisfies . 

A  data  stream  ray        be   defined   as   a   sequence  cf 

values,  existing  over  time  in  a  computation.  If  e  program  is 

executed  in  a  conventional  language,   then   the   history   cf 

successive   values   of   a   variable  forms  a  stream.  Thus,  in 

contrast  to  static  data  structures  such  as  lists  and  arrays, 
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where  ell  the  elements  exist  at  one  tiire,  streams  are 
dynamic  data  structures,  ana  are  addressable  objects  in 
LISP.   Only  the  new  data  received  need  "be   compared   tc   tee 

rules.  .New  matches  are  added  to  the  end  of  the  stream.  A 
temporary  'freezing"  of  the  action  specified  by  an  assertion 
occurs  until  after  the  new  data  is  read  onto  the  appropriate 
streams.  The  stream,  then,  allocs  the  rapid  review  of  stored 
information  witnout  the  penalty  cf  complete  review  of 
potentially  irrelevant  data. 

In  STAMMER2,  confidence  is  provided  in  the  rules  by  the 
creator  of  the  rule.  It  is  calculated  dynamically  on 
request  at  the  time  when  it  is  displayed  to  the  operator 
following  a  rule  match  and  is  not  stored.  Each  assertion 
will  ha^e  as  its  confidence  the  cor  hi  nation  of  the 
confidences  cf  the  rules  and  assertions  which  offer  evidence 
for  it.  A  confidence  calculation  is  highly  dependent  upon 
the  connective  tnrough  which  the  rules  and  assertions  are 
combined.  An  ANE  connective  will,  for  example,  display  the 
confidence  cf  the  smallest  value  of  all  the  rules  combined. 
That  is,  the  conclusion  may  never  be  stronger  than  the 
weakest  piece  of  supporting  evidence. 

During   execution   of  STAivMSB2,   the  user  will   see 

the  following  cycle  repeated  as  long  as  messages  and  reports 

are  received  into  the  message  input  file: 

1.  A  message  revert  is  received,  the  user  is  informed, 
and  the  critical  information  in  the  message  is  printed 
for  reference. 
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c.  A  display  ,  showing  the  arte  situation  with  the  new 
informat  ion  ,  is  drawn.  The  user  may  manipulate  this 
image. 

3.  The  system  makes  some  commentary  on  the  conclusions  it 
Dan  reach,  on  the  basis  of  the  aew  information. 

4.  If  any  conclusions  were  reached,  the  user  is  ~iven  the 
opportunity  to  auery  the  system  about  the  contents  of 
its  data  base. 

The  message  text  is  in  the  basic  format  of  the   received 

ressasre.    A   contact   narre   (if   available),   is  displayed, 

followed  by  location  and  time  of  message.  The  user  vay   pause 

to  examine  this  message  or  select  the  graphic  display  rode. 

Graphical   support    for   STAMMER2    is    provided    by 

TSPLA,   a   software  package  developed  at   the  Naval  Ocean 

System   Center   specifically    for    tactical   situation 

assessment   [Ref.   13].   The  DSPLA  system  is  a  collection  of 

FORTRAN   subroutines   that   allow   storage,   retrieval,   and 

display  of  ship  ana  aircraft  tracks.   This  display,  with  the 

capability  to  shew  maps  with  latitude  and   lon?itude,   nay 

be   manipulated  by  the  user  to  vary  the  scale  as  desired.  As 

scon   as   a   message   is    received   which   contains    a 

demonstrable    assertion   (such   as   a  contact  at  a  specific 

location)  the  display  will  appear   on   the   display   screen 

■if    a   display    is    available   a.r.c        selected   in   the 

initialization  process).  Examples  of  basic  function  keys  are 

are  listed  below. 

M    magnify  about  the  cursor  07   a  factor  of  c 

?.    reduce  about  the  cursor  or    a  factor  of  2 
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V    center  the  view  about  trie  cursor  pcsiticr 
;     set  type  size  to  smallest  on  the  T2KTRCNIX  ^214 
C    return  tc  command  mode 
rcore  detailed  functions  exist,  with  the  full  listing  in  KOSC 
Technical    Document    252.   The   user  may   return  to  the 
command  level  to  begin  the  explanation  and  query  procedure. 
The   explanation   s/ster    provides    two    primary 


capabilities:   retrieval 


memory  and  inference  tracing. 


Although  there  would  appear  to  he  little  difficult/  in 
retrieving  memory  contents  and  tracing  derivations,  the 
display  of  this  information  in  a  human  readable  format  is  a 
-lajcr  cons  idera  t  i  en  .  To  make  the  user  interface  as  natural 
as  possible,  the  explanation  system  provides  a  Query 
language  that  is  "English-like".  This  language  is  an 
extremely  limited  version  of  English,  which  includes  only 
certain  types  of  questions  and  methods  of  phrasing  those 
questions.  However,  the  language  was  designed  so  that, 
while  limited,  it  is  sufficient  to  cover  the  user's  needs 
without  marling  its  shortcomings  apparent.  The  LIS?  function 
ASKUSER,  which  features  recognition  and  prompting,  is 
combined  witn  a  ' p rettypri ater"  (to  add  words  like  'is'  and 
"of"  to  memory  contents)  to  simplify  the  user  interaction. 

A  summary  of  the  queries  which  the  STAMMER2  user  can 
esk,  with  general  examples,  follows.  Following  the 
initial  query  word,  only  tnese  inputs  shown  will  be  allowed 
oj      the   ASKUSIR    function.   The  connective  words  (in  lower 
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case)  will  be  provided  by  the  "prettyprinter"    [Ref.   14]. 

The  words   in  parentheses   may  be  inserted  by  the  user  as 

iesi  red  . 

VHAT  Forrrat:  WHAT  is  (THE  AN  A)   <RELATI0N>   (0?) 

<ITEM> 
ExarpIetWHAT  is  the  COURSE  of  SIGHTINGS 
This   every  provides  ore  of  several  Question 
formats  for  asking  about  entries  in  the  data 
base. 

IS  Format:  IS  (TEE)  <ITEM>  [k   AN  TEE)<RELATION> 

(OF)  <ITEM> 
Example: IS  RADAR  the  SOURCE  of  SIGETING3 
This   question   form   allows  the  user  to  ask 
about  specific  entries  in  the  data  base. 

TILL  re  about   Forrat:   TILL  me  about  <ANYTHING> 

Example:  TILL  me  about  SIGHTING5 
This   is  the  most  flexible  query,  and  allows 
the   user   to   ask   rules,    categories, 
relations,  or  specific  facts. 

WHERE  Format:   WHERE   is   \OBJECT>   or  WHERE   was 

<CiJECT>  at  <TIMI> 

Example:  WHERE  is  CONTACT? 

With  this  command  the  user  can  ask  about  the 

position   of   a   platform,  merchant  lane,  or 
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storm  at  the  present  tine,  or  in  the   future 
or  cast. 


HCW 


WHOSE 


WHC 


Format:   WHY  is  <ASSERTION> 

Example:  WHY  is  A00345 

With   this  command  the  user  can  find  cut  the 

primary  or  immediate  reasons   that   STA^r'ER2 

used   tc   conclude   any   assertion.   All  the 

rules   involved   in   the  decision  will   be 

displayed  . 

Format:   HCW   does   rule   <RULE>   apply   to 

<ASSERTION> 
Example:  EOW  does  rule  ID-LANE  apply  to  A034 
This  query  allows  the  user  to  find  cut  what 
assertions  or  facts  were  involved  in 
permitting  the  rule  to  help  conclude  the 
given  assertion. 

Ecrmat:   WHOSE  <RELATION>  is  <ITEK> 

Example:  WHCSE  TYPE  is  MERCHANT 

This   viery  acts  as  a  partial  inverse  tc  the 

WHAT  iuery. 


Format:   WHO  is  (THE  AN  A-   <RELATI0N> 

<ITEK> 
Example:  WHO  is  INSIDE  IAN  El 


(OF) 


This   provides  another  form  for  querying  the 


data  case . 
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IV.    EXPERIMENTAL  MEiHCLCLCGY 

This  section  presents  a  discussion  cf  exper i^er tal 
methodology  which  may  be  applied  in  the  evaluation  of 
5TAKMER2.  Following  an  overview  of  general  issues,  and  a 
discussion  of  general  levels  cf  experimentation,  three 
specific  stages  of  experiment  are  presented.  vrom  these,  one 
has  been  chosen  to  be  carried  cut  as  a  sample  experiment. 

The  design  of  an  experiment  is  based  on  the  question  to 
be  addressed.  An  experiment  to  compare  systems  with 
different  speeds  of  message  receipt,  for  example,  will 
include  a  message  generator,  an  interval  for  testing,  and 
the  means  to  measure  differences  in  total  messages 
received.  The  criteria  for  "better"  may  be  the  higher  number 
cf  messages  received  in  a  certain  time  or  the  shorter  time 
needed  to  receive  a  certain  number  of  messages.  These 
objective  measures  of  performance  are  established  prior 
to  the  experiment,  with  sufficient  flexibility  to  present 
the  data  to  satisfy  the  question. 

In  more  complex  evaluation,  the  questions  to  be 
answered  ray  be  difficult  to  define.  If  the  goal  of  the 
experiment  is  to  find  the  number  of  messages  received  and 
understood  during  a  definite  time  interval,  the  criteria  for 
understanding  must  also  become  a  part  of  the  experimental 
procedure.    Interpretation   of   the   results   cf   such  an 
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experiment  then  becomes  dependent  on  an  "acceptable"  level 
of  understanding,  an  acceptable  number  of  messages,  or  a 
combination  of  the  two.  This  ray  then  be  an  area  where  the 
objective  measures  ^ive  way  to  less  precise  but  equally 
useful  subjective  evaluation. 

In  order  tc  test  the  suitability  of  an 
artificial  intelligence  system,  measures  of  performance 
ray  be  difficult  tc  establish.  The  components  cf  the 
program  may  be  individually  examined  (the  structure, 
execution,  results),  or  the  overall  improvement"  in  a 
parameter  of  the  function  which  the  program  is  designed  to 
assist  may  be  tested.  However,  to  show  that  the  system 
would  respond  tc  a  question  cr  situation  as  quickly  as 
a  human  decision  maker,  given  that  the  situation  were 
constrained  tc  what  the  program  and  the  r.uman  knew  and  the 
information  flow  was  at  "normal"  speed,  may  prove 
nothing  mere  than  that  computers  are  faster  than  humans. 
Considering  the  difficulty  in  establishing 
meaningful  treasures  of  effectiveness,  subjective  evaluation 
cf  the  capability  and  the  utility  cf  STAMMEH2,  by  tactically 
oriented  individuals,  ray  show  the  greatest  promise  as  an 
evaluation  technique. 

Ixperirenta t ion  may  be  performed  in  stages  ranging  from 
simple  static  debug/validation  experiments,  which 
concentrate  on  the  capabilities  of  the  technology  and 
require   few   resources,    to   more   extensive   technical 


evaluations  cf  S T Af-  MZ?>2  with  a  dynamically  changing  lata 
base  in  s  controlled  environment.  This  could  include 
dynamic  operational  evaluations  in  simulated  ccrrmana  and 
control  environments  pitting  opposing  forces  against  each 
other  under  various  scenarios  using  one  of  the  warfare 
environment  sirulator  U'ES)  war  game  programs  [?.ef.  15]. 

following  an  overview  of  three  possible  levels  of 
testing,  more  detailed  procedures  for  these  levels  will  be 
presented . 

The  "basic  level  of  experiment  consists  of  inputing  into 
a  data  "base  various  static  threat  scenarios  (a  snapshot 
view  cf  the  situation}  concerning  the  status  cf 
'  Piue"  ana  "Orange"  task  forces.  The  test  objective 
is  tc  evaluate  the  capabilities  cf  STAMMERS  tc  read  the  data 
base,  signal  the  existence  of  the  threats,  and  form 
opinions  'based  en  specific  rules  selected  ty  the  test 
conductor;  concerning  other  information  in  the  form  of 
sensor  and  intelligence  reports. 

The  next  level  of  experiment  is  conducted  in  a 
it  ore  realistic  command  and  control  environment  with  a 
dynamically  changing  data  base.  A  detailed  scenario  is 
presented  via  WIS  (or  its  variants),  presenting  a  wide 
range  cf  inputs  tc  STAiv[":E32  under  changing  circumstances 
to  probe  the  limits  of  its  capabilities,  e.g.,  what  threat 
intensity   is   required  tc  "cverlcad"   STAr"hER£,   cr  wnat  is 
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the  length  of  time  fror  the  instant  the  rercrt  is  received 
until  STACKERS  signals   a  conclusion. 

In  the  r:ost  convex  level  of  testing,  WES  is  again  used 
witn  Orar.re  and  Blue  task  forces.  These  fences  ane 
composed  as  closely  as  possible,  within  the  capabilities 
of   available  test   resources,   with  equipment  (sensors, 

weapons,  etc)  end  platforms  that  represent  current  military 
inventories.  Experimental  emphasis  is  shifted  from  a 
technical  evaluation  to  an  operational  evaluation  of  the 
value  cf  STAMMER*  to  tne  decision  maker.  The  cbjective 
is  to  determine  if  STAMMERS  improves  the  commander's  ability 
tc  make  rapid  and  accurate  decisions.  Operationally 
realistic  scenarios  are  the  desired  norm.  Each  scenario 
should  be  replicated  with  different  players  with  sc~e 
trials  having  the  Blue  forces  operating  with  STAMMERS  and 
scne  without.  Operational  measures  cf  effectiveness, 
which  depend  on  the  scenarios  under  play,  are  selected  to 
assess  the  operational  utility  of  STAMMER2.  In  addition,  the 
players  and  the  umpire  team  ere  subjected  to  pest-exercise 
interviews  and  questionnaires  tc  get  subjective  evaluations 
cf  the  worth  of  STAMMERS. 

While  there  are  no  definitive  boundaries  between  the 
levels  cf  the  formal  experiments,  the  range  cf  conditions 
must  be  designed  for  scenarios  that  are  botfc  realistic  and 
informative.  The  actual  analysis  at  each  level  of  experiment 
must  take  into   account   the   limitations   cf   the   scenario 
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generator  r?_ef.  16].  The  three  proposed  levels  of  Experiment 

will  new  be  ires  en  ted  in  detail. 


A.   STAGE-ONE  EXPERIMENTS 

1 .  C  "elective 

Id 

Stage-cne  experiments  evaluate  the  capability  cf 
STAKM1R2  to  identify,  in  a  static  environment,  the  tactical 
situation  that  confronts  an  afloat  task  force  and  tc 
describe  the  situation  semantic ally  and  graphically.  This 
type  cf  experiment  establishes  the  type  of  threats  that 
STAMMERS  car.  evaluate  and  assesses  the  timeliness  of  the 
warnings.  The  experiments  may  also  serve  to  provide  feedback 
through  which  STAMMERS  may  be  refined. 

2 .  Resources  Required 

a .  Technical 

The  STAMMERS  program  and  associated  computer 
hardware  are  required  for  this  experiment.  At  present, 
STAMMER2  is  resident  on  the  TCPS20  system  at  NOSC,  San 
D  i  e  g c  ,  California. 

b.  Display 

An  alphanumeric  display  system  will  be  required 
for  players  and  the  umpire  team.  In  addition,  a  graphical 
display  of  the  situation,  as  is  normally  generated  'cy 
WES  rust  be  available  tc  compare  it  with  that  presented  by 
STAMMER2. 
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c  .   S c enar ic 

The  tactical  scenario  is  presented  as  e  series 
cf  operationally  realistic  "data  plates'  constructed  in 
advance  cf  the  experiment.  Each  data  plate  is  a  -lata 
snapshot  describing  a  tactical  situation  at  a  given  tire. 
The  plate  should  include  various  information  (course,  speed, 
sensor  reports,  intelligence)  available  to  a  Tactical  Action 
Officer  tc  describe  the  situation  fully. 
d  .   Personnel 

The   personnel   required   for   the   experiment 
include   observers   of   the   output   (threat   assessors),   5 
scenario  controller,   and  a  test  director, 
e.   Support  Program 

In  addition  tc  scenario  generating  requirements , 
software  tc  measure  and  record  the  times  at  which  the 
messages  are  output  ana  the  tires  designated  by  the 
observers  is  required.  This  program,  based  on  tne 
ARFANET  TYPESCRIPT  feature,  records  all  inputs  and 
outputs  tc  selected  terminals. 
3.   General  Context 

a .   Concep  t  and  Need 

STAivMLF.£  is  being  developed  to  interact  with  a 
^rapnics  display  system  to  support  the  corrrander's  decision 
processes.  In  order  tc  assess  the  value  cf  the  technology 
*e  must  determine  its  capabilities  and  shortcomings.  For 
STAMMER2  tc  be  useful  tc  a  decision  maker  it  must  be  able  tc 


evaluate  threats  by  a  comparison  end  collation  of  all-source 
reports  and  display  the  situations  in  a  tirely  manner. 

b.   General  Situation  and  Scenarios 

A  suitable   number   of   data   plates   should   be 

developed   to   vary   the  tactical  situation  significantly  in 

both  the  complexity  of  the  situations   and   the   nature   and 

number   of   the  platforms   involved.  A  data  plate  should  be 

selected   at    random,    its    number    recorded    and    the 

appropriately   stored   data  input  to  the  file  which  STAMMER2 

will  read.  An  observer,  unaware  of  the  contents  of  the   3ata 

plate,   [Tcnitors   the   contents   of   the  display   terminal, 

evaluates   the   information   received,   and   describes    the 

situations  as  he  sees  them. 

4 .   Evaluation 

a .  Da  ta  collection 

The  following  aata  should  be  collected  for 
each  trial:  the  data  plate  identification  number,  the 
situation  assessment  of  tne  evaluatcr,  and  the 
situation  assessment  given  by  STAMMER2. 

b.  Analysis 

The  primary  areas  of  concern  for  the  stage-one 
experiments  are:  a  comparison  of  the  STAMMSR2  output  with 
tne  human  assessments  of  the  threats,  and  a  determination 
of  the  times  required  for  STAMMEP.2  to  process 
different   types   of  situations. 


Post-test  anal/sis  will  exariae  the  data  sheet 
tc   ascertain  if   STAmTER2  accurately  evaluated  or  described 

and  displayed  all  phase-one  situations.  This  is  compared 
with  the  eveluators'  ability  tc  do  the  seme  based  or  the 
same  data. 

c.   Anticipated  Results 

It  is  anticipated  that  STAPM2H2  will  signal  the 
existence  of  tnreats  with  very  little  time  delay  and  that 
the  situation  end  display  will  be  presented  accurately.  Tor 
those  cases  where  ccmplex  multiple  situations  are  present 
simultaneously,  the  tests  may  indicate  en  overcrowding  of 
the  display  and  a  lack  of  time  to  exercise  the  logic  trace 
capability  cf  STAMMER2. 

c .   Comments  and  Special  Instructions 

Each  experimental  subject  will  go  through  a  shcrt 
STAGERS  indoctrination  session  during  which  ne  is  given  a 
summary  of  tne  theory  and  implementation  cf  STAGERS,  a 
"threat"  briefing  tc  explain  the  scenario,  and  an  overview 
cf  what  is  expectec.  of  hirr.  Care  should  be  taken  tc  include 
in  the  data  plates  as  many  situations  as  possible  that 
STAfM2E2  has  been  designed  to  handle  in  realistic 
situations.  It  is  recommended  that  several  different 
operators  be  subjected  to  tne  complete  set  of  plates  and  be 
asked  tc  subjectively  evaluate  the  displayed  information  tc 
provide  information  which  may  be  useful  in  the  design  of 
fcllow-on  display  experiments. 
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The  sta^e-cne  experiT-er.tat  ion  discussed  above  will 
need  to  be  carried  out  in  the  course  of  e  full  er.d  c?reful 
evaluation  of  STAP-.MP.2  as  a  tool  In  tactical  assessrent.  For 
the  purpose  of  this  thesis,  however,  only  the  stage-twc 
experiment  (as  discussed  below;  v*as  actually  performed. 

P.   STAGE-TWO  EXPERIMENTS 

1 .  Objective 

The  objectives  cf  stage-twc  experiments  are:  tc 
assess  the  technical  capability  of  STAMMFR2  actions  on  a 
dynamically  changing  data  base  generated  by  creating 
threat  situations  through  the  exercise  of  WES  (or  its 
variants)?  tc  determine  the  limits'  of  the  capability 
of  STAN\N'ER2;  and  to  obtain  subjective  opinions  about 
STAMMER2. 

2 .  Resources  F.eauired 
a  .   Techni  cal 

The  STAMMER2  program   and  associated  computer 
hardware   are   required   for   this   experiment.   The  Warfare 
Environment  Simulator  (VIS)      program   is   available   on   the 
TEN'EX  system  at  NOSC. 
b.   Display 

An  alphanumeric  display  system  is  required  for 
players  and  the  umpire  team.  The  GENISCO  system,  a  color 
graphic  display,  xey  be  used  tc  present  the  situation 
which  is   generated  by  WES  . 
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c .   Scenario 

An  operationally  realistic  scenario  is 
generated  to  test  STAMMSE2  in  a  tactical  situation  at  a 
given  tine.  The  scenario  should  include  various 
information  (course,  speed,  sensor  reports,  intelligence) 
available  to  a  Tactical  Action  Officer  to  describe  the 
situation  as  it  is  aorrnally  presented  by  a  IS  ,  via 
visual  display  and  status  boards.  The  e valuator  will  be 
allowed  to  interact  with  units  under  his  "control"  by 
ordering  course  and  speed  changes,  and  enabling  various 
sensors  available  to  hirr, .  The  test  director  nay  insert 
varicus  elements  of  intelligence  as  desired, 
d  .   Personnel 

The   personnel   required   for   the   experiment 
include   otservers   of   tne   output   (threat   assessors),   a 
scenario  controller,   and  a  test  director, 
e.   Support  Prcsram 

In  addition  to  scenario  generating  requirements , 
software  to  measure  and  record  tne  tires  at  which  tne 
messages  are  output  and  the  tines  designated  bv  the 
observers  is  desirable.  This  program,  based  on  the 
ARPANET  TYPESCRIPT  feature,  records  all  inputs  and 
outputs   to   selected  terminals. 
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f.   Evaluation  questionnaire 

Following  a  stud/  of  the  measures  cf  operational 
effectiveness  which  may  te  achieved  in  this  interactive 
experiment,  -r  evaluation  questionnaire  should  be 
constructed  to  establish  the  level  to  which  these  measures 
nave  teen  satisfied. 

3.   general  Context 

a.   Concert  and.   Need 


A   complete   technological   assessment   o: 


capability  of  STAM(SER2  includes  &r.  evaluation  in  a  realistic 
environment  with  a  dynamically  changing  data  base.  The 
experiment  must  subject  STAMh3R2  to  a  wide  of  situations 
under  varying  levels  cf  sensor  activity  in  an  effort  to 
exercise  all  capabilities  of  STAH-IR2  and  try  to  probe  the 
"limits"  of  those  capabilities.  These  limits  could  be  the 
size  of  data  base,  number  of  reports,  or  frequency  cf 
reports  . 

b.   General  Situation  and  Scenarios 

'*'ES  scenarios  will  be  generated  to  present  a 
variety  of  sensor  reports  5vA  levels  of  activity  by 
platforms.  An  evaluator  familiar  with  tactical  war  gaming 
will  observe  the  play  cf  the  .^ame  wnile  exercising  STAMMER2 
to  the  maximum  extent  possible. 

Each  experimental  subject  will  go  through  a 
short  STAMMER2  indoctrination  session  during  which  he  will 
be  given  a  summary   cf   the   theory  and   implementation   cf 
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STArtfER2,   e   "threat"  briefing  to  explain  the  scenario,  and 
an  overview  of  what  is  expected  cf  birr . 

During  the  course  cf   the  actual   experimental 

run,   the  evaluator  nay  be  assisted  by  the  VIS  operators  and 
test  director  in  the  mechanism  of  input,  output,  and  display 
to  rerove  extraneous  or  distracting  details  of  the  game. 
4 .   Zvaluat ion 

a.  Tata  Collection 

The  following  data  should  be  oollected  for  each 
scenario:  the  times  at  which  the  data  oase  is 
updated,  the  situation  analysis  by  STAMMERS,  and  the 
situation   analysis  ~z;j      the  operator. 

These  data  will  be  augmented  by  the  situation 
assessments  -^ade  ty  tne  test  director  who,  having  become 
familiar  with  the  full  scenario,  can  assess  the  "ground 
truth"  by  having  knowledge  cf  the  actions  of  ail  platforms 
throughout  the  scenario.  The  questionnaire  ^iven  to 
each  operator  should  consist  of  questions  designed  to  assess 
the  realism  of  the  scenario  and  the  utility  cf 
STANMER2   as  objectively  as  possible. 

b.  Analysis 

Comparison  is  made  between  the  situation 
descriptions  by  STAMMER2,  the  evaluators,  and  ground  truth 
to  assess  the  general  accuracy  of  these  descriptions.  The 
questionnaire  results  ray  be  presented  in  tabular  fcrr  and 
the  objective  comments  discussed  in  the  conclusions. 
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c.   Anticipated  Results 

While  it  may  be  possible  to  generate  saturation 
points  for  5TAMMER2,  manifest  in  tire  delays  wherein  the 
STAf'MER2  display  significantly  lags  behind  the  >.TS  scenario, 
the  serial  nature  cf  report  processing  would  seeT  to 
preclude  this  possibility,  providing  the  query  function  of 
STAf*.M5R  2  is  not  extensively  usea. 

t.      Comments  end  Special  Instructions 

■  Ml— ■■      I.       II    —  —  ■■-.■■■■■^         -III   -       .1     Mil  — — — 

The  scenario  tape  should  be  maintained  so  that  a 
eiven  ga~e  may  be  replicated  exactly  with  zero  variance  for 
different  operators  and  so  that  the  test  director  can 
thoroughly  determine  the  situations  as  they  exist  (or  are 
abcut  to  exist,.  Having  the  sames  en  tape  would  allow  trials 
without  WES-trained  operators  to  input  the  instructions 
required  by  a  detailed  script.  In  addition,  an  observer 
could  monitor  the  Blue  forces  in  one  or  rore  of  the 
scenarios  in  a  run  without  STAPIMER2  and  make  comparisons 
with  the  exact  run  without  STAMMERS.  Finally,  if  the  runs 
were  net  available  en  tape,  very  detailed  scripts  describing 
all  of  the  actions  of  both  the  Blue  ard  Orange  forces  might 
be  written  for  each  of  the  scenarios  to  accomplish  the 
obiectives  of  this  test. 


tc 


C.   STAGE-THREE  EXPERIMENTS 
1 .   Objectives 

The  sta^e-three  experiment  has  as  its  objectives   to 
provide  an  operational  evaluation  of  the  military  utility  of 

STAMMERS  to  a  decision  maker  at  en  afloat  task  force  command 
and  control  center  in  a  simulated  environment  and  tc 
evaluate  whether  STAMMER2  improves  the  decision  maker's 
atility   to  make   rapid   and  accurate  decisions.   As  in  the 


case   of   the   stage-one   experiment,    this   level 


of 


experimentation  are  discussed  nere  but  v,ere  act  performed  as 
part  of  this  research. 

c  .   Resources  Required 
a .   Technical 

The   STAMMER2  and  WES  programs    and   associated 
computer  hardware  are  required   for   this   experirent. 
b  .   Zisplay 

An   alphanumeric  display  system  is   required  for 
each   tea^  of  players   and   the  umpire  team.   The  GENISCO 
system,  a  color  graphic  display,  may  be  used  to  present  the 
situation  which  is   generated   by  WES  . 
c  .   Scenario 

An  operationally  realistic  scenario  is 
generated  to  test  STAMMER2  in  a  controllable  tactical 
situation.  The  scenario  should  include  various 
information  (course,  speed,  sensor  reports,  intelligence) 
available  to  a  Tactical  Action  Officer    tc   describe    the 
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si  tret  ion  as  it  is  normally  presented  by  WIS,  vie 
visual  display  and  status  boards.  The  tears  will  be  allowed 
tc  interact  with  units  under  their  command  by  cr^.erirr- 
course  and  speed  changes,  enabling  various  sensors,  and 
executing  tactical  orders  against  an  opponent.  The 
umpire  may  insert  various  elements  of  irtelligence  as 
desired  . 

d  .   Personnel 

The  personnel  required  for  the  experiment 
include  observer  tears  to  act  as  Slue  and  Grange 
forces,  a  scenario  controller,  an  umpire  tear,  and  a  test 
di  rector . 

e .  Support  Program 

In  addition  to  scenario  generating  requirements, 
software  to  measure  and  record  the  times  at  which  the 
messages  are  output  and  the  times  designated  by  the 
observers  is  required.  This  program,  based  en  the 
APPANET  TVPESC?IFT  feature,  records  all  inputs  and 
outputs   to   selected  terrinals. 

f.  Evaluation  Questionnaire 

Following  a  study  of  the  measures  of  operational 
effectiveness  which  may  be  achieved  in  this  interactive 
experiment,  an  evaluation  questionnaire  should  be 
constructed  to  establish  the  level  to  which  these  measures 
nave  teen  satisfied. 
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3  .   General  Context 

a  .   Concept  and  Need 

It  is  possible  that  the  value  cf  STA;VM?.2  as  a 
decision  aid  ray  only  be  assessed  in  an  operational 
scenario.  Jcr  example,  it  ray  enable  a  decision  maker  to 
better  understand  the  threat  situation  which  faces  his 
forces  and  to  make  ^ore  accurate  and  timely  decisions. 
These  experiments  would  pit  two  opposing  decision  rakers 
against  each  ether  in  a  simulated  ocnTand  and  control 
environment  and  seek  to  measure  the  value  of  STAMMIR2  by 
comparing  the  decision  Takers'  performance  operating  with 
and  without  STAMMIR2. 

b.   General  Situation  and  Scenarios 

The  operational  evaluation  of  STAMMEE2  requires 
operationally  realistic  scenarios  consisting  of  31ue  and 
Crer.ge  task  forces,  which  would  be  acted  out  with  each  tear 
given  the  flexibility  cf  exercising  complete  control  over 
their  forces  as  long  as  they  do  not  countermand  their 
ordered  missions.  7cr  each  scenario,  an  operational  measure 
cf  effectiveness  (MOE)  or  multiple  MOEs  should  be  selected 
and  used  to  assess  the  operational  utility  of  STAN'MR2. 

Because  of  the  free-play  flexibility  given  to 
the  Blue  and  Grange  force  commanders,  the  war  games  and  the 
resulting  outcomes  may  vary  significantly  from  trial  to 
trial.  Consequently,  each  scenario  should  be  replicated  in 
such  a  way  as  to  avoid  foreknowledge  by  a  playing  team.  This 
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za^  be  done  by  usint-  a  nurber  of  teams  ("based  on  the 
availability  of  assets},  with  multiple  scenarios.  By  varying 
the  use  and  non-use  of  STANKER2  by  a  tear,  (considering  that 
a  tear  is  expected  tc  play  mere  than  one  game),  a  comparison 
of  performance  based  on  the  MCZs  may  be  made. 

Eacn  tea^  should  go  through  a  brief 
indoctrination  session  luring  which  the  following  points 
will  be  discussed:  the  theory  and  inplemente t i on  of 
STAMMER2,  a  "threat"  briefing  to  explain  the  scenario  and  an 
overview  of  the  mission,  and  an  overview  of  the  measures  of 
effectiveness. 

4 .   Eva lua t ion 

a.   Data  Collection 

The  following  data  should  be  collected  for 
each  scenario:  tirres  at  wnich  the  data  base  is 
updated,  situation  assessments  by  STAMM5R2,  situation 
assessments  by  the  tearrs,  actions  taken  qj  the  decision 
maker,  and  data  required  tc  determine  the  selected 
operational    reasures    of  effectiveness. 

These  data  may  be  au-p-ented  by  the  situation 
assessments  made  by  the  test  director  who,  having  become 
familiar  with  the  full  scenario,  can  assess  the  'ground 
truth"  by  having  knowledge  of  the  actions  of  all  platforms 
throughout  the  scenario.  The  questionnaire  given  tc 
each  meratcr  should  consist  of  Questions  designed  to  assess 
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the   realisrr   of   the   scenario   end    the   utility   of 

STAK-IR2   as  objectively  as  possible. 
b .   Analys is 

Data  summaries  rray  be  made  for  each  scenario  and 
tabular  displays  should  te   rrace    cf  the  MG2s . 

A  correlation  analysis  should  be  reds  with  the 
time  sequence  of  actions  taken  by  the  decision  maker 
based  on  the  situation  assessments  made  by  STAMMER2  to  try 
to  determine  hew  much  the  commanders  utilize 
STAPMIR2  and  what  information  is  most  useful  to  them  A 
comparison   cf   the   game  outcomes,   as   reflected   by   the 


CIs,   may  be  made  using  analysis  of  variance   to   see 


if 


there   is   any   significant  difference  due  to  STAfMER2 . 

A  comparison  may  be  male  of  the  game  outcomes 
and  the  subjective  appraisals  of  STAMMSR2  to  see  if  there 
was  a  relationship  between  how  well  the  forces  performed  and 
hew  much  they  liked  STAMMSE2. 

c  .   Anticipated  Results 

It  is  anticipated  that  there  could  be  a  lar^-e 
variance  in  the  overall  end-  cf-game  operational  measures  cf 
effectiveness.  The  possibility  exists  that  the  MOFs  will  be 
sufficiently  ambiguous  that  any  signal  due  tc  STAivMER2  may 
not  be  discernible.  Thus,  the  subjective  evaluations  of  the 
teams  "ra~j  provide  the  only  discrimination  in  the  test,  with 
no  quantitative  assessment  of  the  value  of  STAMMIR2 
possible . 
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The  correlation  analysis  of  the  decision  maker's 
actions  and  the  STAMMERS  assessment  should  reveal  the  type 
cf  information  that  is  most  useful  to  a  decision  maker  end 
how  much  he  jay  grow  to  rely  on  STAMMER2.  Overall,  these 
experiments  should  yield  important  information  ecout  the 
acceptability  of  STAMMZR2  to  the  decision  maker  and  its 
utility  to  him. 

5.   Comments  and  Special  Instructions 

The  usefulness  cf  STAMMER2  tc  a  decision  maker  may 
strongly  depend  on  the  manner  in  which  STAMMIR2  outputs  are 
displayed  tc  the  Blue  forces.  This  ^i^ht  he  especially  true 
in  high  density  situations  with  a  backlog  of  unprocessed 
sensor  reports.  The  nature  cf  the  display  dedicated  tc 
STAMMER2  should  be  considered  prior  to  stage  three 
experiments.  The  use  of  a  terrinal  other  than  the  graphic 
display  terminal  for  auery  and  response  actions  allows  the 
picture  tc  remain  undisturbed  until  refinement  of  scale  or 
sequential  processing  is  requested.  The  display  system  is 
an  integral  part  of  STAMMER2  so  the  evaluation  should  be 
based  en  the  nest  flexible  display  available. 

A  tape  record  of  the  game  should  be  made  of  the 
inputs  and  outputs  of  ea:h  game  so  that  evaluation  personnel 
can  reproduce  a  given  trial  and  conduct  a  review  cf  any 
point  tc  assist  in  "what  if"  types  cf  analysis. 
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The  scenarios  used  should  te  operationally 
realistic,  and  the  starting  conditions  end  measures  of 
effectiveness  rust  be  selected  to  exercise  the  full  range  of 
STArNER2  capabilities  and  tc  yield  useful  information.  The 
scenarios  reed  not  te  developed  just  for  the  purpose  of 
exercising  STAMMEH2.  Instead,  the  Blue  and  Crange  task 
forces  should  reflect  as  closely  as  possible,  within  the 
ccnstraints  cf  resources,  the  composition  of  fcrces 
currently  available  or  projected  as  desired  and  the  types  of 
rissions  normally  undertaken  by  each. 

The  subjects  used  in  the  experiments  nay  be  very 
important.  For  purposes  cf  scientific  credibility,  the 
subjects  Tust  nave  command  and  control  experience  at  high 
levels,  preferably  Navy  Captains  (0-6)  or  Admirals.  A 
potential  criticises  of  experiments  which  attempt  tc 
determine  the  operational  utility  of  a  system  is  that  the 
'operators'  aid  not  reflect  the  potential  user  population. 
This  would  seen  especially  true  when  the  evaluations  depend 
heavily  on  subjective  appraisals.  There  are  practical 
difficulties  in  obtaining  such  test  subjects.  However,  the 
issue  is  whether  these  tests  can  be  viewed  as  scientific 
experiments  cr  demons traticns  without  the  operational 
realism  engendered  by    their  presence. 
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V.   SAMPLE  EXPERIMENT 

a.   DESCRIPTION 

A  sample  experiment  was  selected  from  the 
preceding  discussion   tc   illustrate  both  the  capability  of 

STAMMER2  and  and  the  practical  application  cf  these  rethcds. 

The  stare-two  experimental  methodology  was  selected  for 
this  thesis.  Selection  criteria  included  the  expertise  cf 
the  test  subjects  available  and  the  availability  cf  the 
facilities  at  the  Naval  Postgraduate  School.  The  test 
subjects  selected  were  military  officers  with  background  in 
commend  and  control  concepts  gained  through  formal  education 
and  a  wide  range  of  military  experience.  The  Command, 
Control,  and  Communications  Laboratory  contains  extensive 
graphic  and  text  capability,  and  the  STAMMEF2  and  WIS 
software  are  available  at  the  Naval  Ocean  Systems  Center 
(NOSC),  San  Diego,  California. 

The  Warfare  Environment  Simulator  (WES)  war  game 
rrccedure  involves  the  construction  of  a  computer  file  which 


cntains  identification 


all   desired   players,   initial 


positions,  courses,  and  speeds.  Flatforms  may  be  selected 
from  a  resident  data  base  which  assigns  senscr  and 
weapons  configurations  to  the  units.  Following  game 
initialization,  the  units  may  be  maneuvered  en  command  *rcm 
the   player    through  a   computer  terminal,   with    display 
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of  exact  geographic  location  of  known  units  available  on 
selected  graphic   display  monitors.    Interaction   cf  the 

units  rray  "be  ordered  by  the  players  (as  in  attacks),  but 
sensor  reports  are  dependent  or  detection  algorithms  in 
fcSS.  7or  example,  detections  oj  a  surface  search 
radar  against  another  surface  vessel  occurs  at  2?  File 
minimum  separation.  Sensor  reports  are  given  to  the  player 
via  a  status  board  text  display  and  on  the  geographic 
display  which  is  updated  every  ga^e  minute. 

This  experiment  was  not  interactive  in  that  the 
operator  did  net  influence  the  series  cf  events  which 
were  being  generated  by  WIS.  It  was,  however,  interactive 
in  the  sense  that  the  operator  nad  the  capability  tr  select 
different  status  board  displays  and  change  range  scales 
as  desirea.  The  fully  interactive  war  garre  as  suggested  in 
the  third  level  cf  experiment  v,culd  allow  operator 
influence  of  the  event  stream.  The  prepared  scenario 
used  in  this  experiment,  which  included  initial 
positions,  initial  courses  and  speeds,  and  orders  to 
the   various  units  are  included  in  Appendix  I. 

The  scenario  for  this  experiment,  venerated  in 
WIS,  consisted  of  three  U.S.  warships  conducting  a 
surface  surveillance  of  a  merchant  lane  in  the  vicinity  of 
the  North  Atlantic,  with  neutral  ard  ^potentially )  hostile 
platforms  in  the  area.  The  warships,  assisted  by  patrol 
aircraft   conducting   similar   surveillance,  were  to  locate, 
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track,  and  identify  ctr.er  units  in  the  ~ree.  Th€ 
experimental  subject,  'embarked"  on  the  cruiser  Belknap,  had 
STArMER2  available  to  assist  in  that  mission.  The  STAhMZR2 
startup  procedure  is  given  in  Appendix  A. 

The  scenario  was  chosen  based  on  the  range  cf  situations 
available  and  the  relative  realisr  of  a  nc n-combat ive 
confrontation.  The  scenario,  tactics,  sensors,  ari  sensor 
characteristics  are  purely  hypothetical.  The  data  base 
[Appendix  B]  ard  rules  [Appendix  C]  are  assumptions  tailored 
to  this  scenario,  cut  are  easily  translatable  to  those  which 
a  Tactical  Action  Officer  would  recognize  as  having 
potential  applications.  The  briefing  cf  the  subjects 
included  ar  explanation  cf  the  artificialities. 

A  difficulty  in  the  actual  conduct  of  the  experiment 
existed  due  to  tne  nature  cf  the  reports  on  which  STAHMER2 
operates.  The  sensor  reports  received  by  an  operator  of  WES 
are  both  visual  cues  en  a  graphics  display  '.lines  cf  bearing 
or  new  platform  symbology),  and  in  a  table  of  detections 
(the  Electronic  Warfare  ctatus  Scard,  the  Surface  Status 
Board,  or  the  Air  Status  Board).  The  form  of  the  tabular 
repcrt  is  formatted  as  to  its  numerical  sequence, 
identification,  and  location  (if  applicable;.  The  reports  as 
generated  by  WIS  are  net  readable  by  STAMMZR2.  One  example 
cf  the  disparity  is  that  STAMMER2  recognizes  latitude  and 
longitude  in  degrees  and  tenths  rather  than  degrees  and 
minutes.  The   LIS?  version  cf  WES,  LWES ,  dees  repcrt   in   the 


appropriate  format,  but  the  difficulty  ir  using  LWSS ,  such 
as  complex  scenario  generation  and  limited  cut  jut 
capability,  far  outweigh  this  advantage. 

The  solution  to  the  forrat  disparity  lies  in  the 
nature  of  WES .  ta  i  th  duplicate  platforms,  starting  at  exact 
locations,  giv^  precisely  the  same  order  of  instructions 
(such  as  course  changes  or  sensor  status),  sensor  reports 
are  also  duplicated.  That  is>.  the  detections  are  based  en 
maximum  ranges  of  detection  v* hie h  are  unchanged  and 
which  are  combined  in  the  aZS  detection  algorithms  in  the 
same  way,  to  produce  duplicate  reports  at  the  same 
ga^e  time.  This  characteristic  was  exploited  by  running  a 
game  to  a  suitable  point  in  time  using  the  ARPANET 
typescript  feature  which  copies  all  terminal 
interactions  as  they  occur.  This  typescript  shows  the 
various  reports  received  during  a  gare,  complete  with  the 
time  cf  receipt.  Synthesizing  game  reports,  a  complete 
file  of  reports  [Appendix  D]  was  built,  in  the  forrat 
from  which  STAMMER2  could  read  its  input  data.  Therefore, 
STAr'MER2  was  net  receiving  direct  output  from   the  sensors, 


but   via 


intermediate  nrccess  net  uart  of  either  WIS  cr 


STAPMER2. 

STA^MER2,  using  the  TSKTRCNIX  4£14-1  terminal  for  both 
text  and  graphic  display,  was,  in  effect,  a  separate  entity 
running  concurrently  with  the  gare.  The  comparison  of  the 
subject's   perspective  of  the  situation  and  that  of  STAMKEB2 


r-* 


is  still  valid  only  as  Ion,-   as   simultaneous   retorting   is 
carefully  ^ainta  ined . 

The  artificiality  of  the  message  input  could  have  biased 
the  experiment  tc  a  remarkable  ce^ree  hut  STAMMEE2  has 
one  feature  which  emelioratFS  this  problem.  STAMPER*  ices 
not  process  the  next  sequential  report  in  the  file  until 
the  user  quits  the  auery  rode  of  operation.  That  is, 
until  the  quit'  ccrrrand  is  given  there  is  no 
processing  of  the  next  sequential  report.  Pecause  the 
experimental  subject  is  net  expected  tc  he  the  actual 
operator  of  the  STAMMER2  hardware,  the  operator  can  he 
instructed  net  tc  leave  tne  euery  mede  until  the  appropriate 
game  time  (which  is  displayed  on  the  input/output  terminal 
for  r*ES) . 

relieving  a  suitable  period  of  game  time  where  the 
subject  is  encouraged  to  exercise  STAMMER2  a  questionnaire 
was  presented  to  gather  the  impressions  and  appraisal  of  the 
experiment  in  general  and  STAMMERS  in  particular. 

E.   RESULTS  AN2  CONCLUSIONS 

The  questionnaire  supplied  to  test  subjects,  with  a 
summary  of  results,  is  listed  below.  The  intent  cf  the 
questions  was  to  verify  the  "realism"  of  the  scenario  and 
to  gather  a  subjective  evaluation  of  STAMMIR2.  Tne  scoring 
scale  of  1  to  12  was  arbitrarily  chosen,  with  12  the 
rost   positive  or  most   favorable   response.   The   test 
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sutjects  consisted  cf  eight  Navy  and  Air  Force  officers, 
ranging  in  grade  from  0-2  to  0-5,  who  ere  renters  of 
the  Command,  Control  and  Communications  curriculum  at  the 
Naval  Postgraduate  School,  Ponterey,  California.  All  have 
beer  expesed  tc  decision  tnecry  ard  tc  an  overview  cf  the 
naval  tactical  environment.  The  Kevy  participants  included 
three  C-4s  and  cne  0-3.  All  cut  one  c^  the  Navy  0-4s  had 
Tactical  Action  Officer  training. 

STAKMIR2  QUESTIONNAIRE 


1.    Prior   to  the  scenario  presentation,  was  the  purpose  cf 
STAKMER2  adequate!/  explained? 


Responses 

Average   6.12  5 


9    10 


3 


£.    Was   the   scenario    representative   cf   your  previous 
exrerience  with  WIS  or  LWES? 


u 


Responses 


7 


10 


3.   Was  the  scenario  realistic? 
12    3    4    5 
Responses  1 

Average   6.375 


9 
3 


ie 


tc 


4 .  >.  e  r  e  the  displays  consistent  with  the  sense?  repents? 

i254ee?E9i: 

Responses  223     1 

Average   8.375 

5.  '/as   sufficient  time  available  to  understand  the  sensor 


■epcrts? 


7 
2 


12  3  4  5  5 
Responses  1  1  1 
Average   7.5 

6.  Were  the  STAMftER2  graphics  clear? 

12  3  4  5  6 
Responses  1  13  2 
Average   4.5 

7.  Was  the  STAMMER2  commentary  understandable? 

12     3     4     5     6     7 
Responses  1         3    1 

Average   5.875 


9 

2 


10 

1 


12 


1? 
1 


8.    Were   the   STAMMER2   conclusions   consistent   with  your 
conclusions? 


12     3     4     5  5 

Responses  2 
Average   7.25 

9.   Eid  you  use  the  explanation  traci 

12     3     4     5  6 

1 
Average   8.375 


9 
1 


1? 


Responses 


12 
1 
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10.   If   the  explanation  trace  was  used,  were  the  assertion61 
consistent  with  the  conclusions? 


1     2  3     4     5     6     7     f 

Responses  2    3    : 
Average   7.375 
11.   Was  the  presence  of  STAMK5R2  distracting? 

12  3     4     5     6     7* 

Responses  1    3  2 
Average   4.75 


10 


ie 
i 


1<£  . 


Would      an      assistant   who   would    filter   information   frop- 
STAP'MSR2    to    the  minimum   essential    level    be   useful? 

12345679912 

Pesronses  11  3  3 

Average        S.5 

13.   Is  STAMME52  useful? 

123456789     1? 

Responses  1         1    4    2 

Average   7.75 

The  results  shew  an  overall  trend  towards  an  average 
response  which  could  be  considered  favorable,  in  the 
vicinity  of  £.  The  exceptions  provide  a  clear  contrast 
consistent  with  conditions  noticed  during  the  experiment. 

Question  5  had  the  lowest  average  ranking.  The 
difficulty  encountered  with  the  display  may  best  be 
described  by  shoving  the  aisplay  at  three  selected  gare 
minutes.  figure  1  is  the  last  display  without  E'a 
bearing  lines,  which  occurs  at  minute  3.   Figure   2   is   the 
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display  at  .?ame  minute  14,  which  includes  intelligence  data 
rM?.l  end  MIR2),  surface  contact  reports,  E¥  bearing  lines 
and  the  location  of  friendly  units.  Figure  ?  is  the  display 
at  game  minute  45  and  is  a  compendium  cf  all  reports 
received  to  that  point.  Close  observation  of  the  screen  as 
the  graphics  were  being  drawn  shewed  a  sequential  entering 
cf  the  information  as  it  was  being  drawn.  Sere  reasure  of 
crder  was  discernible  if  this  process  was  carefully 
followed,  however  the  display  is  unreadable  after  it  is 
completed . 

A  modification  to  the  display  package  is  required.  Cne 
suggestion  fror  the  test  subjects  was  that  only  the  last  two 
occurrences  of  a  contact  be-  displayed,  with  earlier  reports 
available  for  historical  review  as  directed  by  function  Y.ej 
selection. 

A  benefit  of  STAMMER2  was  demonstrated  during  the 
experiment.  In  every  iteration  of  the  display,  intelligence 
information  in  the  form  of  contact  reports  received  from 
external  sources  was  present  on  the  screen  as  a  reminder 
to  the  decision  maker.  Because  a  suspension  concerning 
this  information  is  created  in  memory,  this  information  was 
available  at  all  times  for  review  and  explanation  as 
desired.  This  information  was  not  subject  to  update  in  this 
scenario  and  did  not  seem  to  contribute  to  the  general 
clutter  discussed  above. 
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Fig.    3      STAFMIR2   Graphics    Display,    Sare   Minute   45 
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There  appeared  to  be  a  disparity  between  the  STANM5R2 
conclusions  dnci  those  of  the  test  subjects  (questions  £  end 
10).  The  difficulty  was  most  apparent  in  cases  where  the 
test  subjects  were  able  to  cross-fix  passive  detection 
tearing  lines  tc  a  geographic  position.  Rather  than  conclude 
that  an  emitting  platform  exists  at  that  point  of 
intersection,  STAKMER2  would  continue  to  Take  conclusions 
based  only  on  the  individual  FW  reports.  Here  the  major 
difficulty  appeared  tc  be  in  the  ccrstruc  tier,  of  STAMM5R2 
rules  rather  than  the  functioning  of  the  rules  that  exist. 
A  rule  correlating  or  position-fixing  multiple  passive 
detections  of  the  same  emitter  would  appear  to  be  a 
relatively  simple  matter. 

The  explanation  trace  feature  of  STAMMER2  proved  to  be 
valuable   as   both  a   decision  aid  and  as  a  training  tool. 

While  the   users   commented   (question   9)   that   the   trace 

v. 

included  extraneous  information,  the  level  of  understanding 
of  the  scenario  increased  markedly  due  to  The  increased 
appreciation  of  what  rules  were  being  applied  and  how 
STAMMER2  applied  them. 

The  low  average  score  for  question  11  does  not  fit 
the  general  pattern  of  the  euest ionnaire  due  to  the 
wording  of  the  question  .  The  negative  response  indicates 
that  STAMM2R2  was  not  distracting  to  the  user,  ^he  potential 
exists  for  the  test  subjects  to  became  engrossed  in  the 
execution  of  STAMMSS2  and  the  explanation  trace  to  the  point 


where  the  conduct  cf  the  game  becomes  a  Tircr  consideration. 
This  apparently  was  not  the  case  here. 

The  remainder  of  the  avera&es  appear   to   be  generally 

consistent  with  favorable  response  to  both  the  scenario  in 
general  and  to  STAMMER2  in  particular. 

This  experiment  depends  heavily  en  subjective 
evaluation.  While  the  scenario  and  operation  of  STAMMER2 
appear  to  have  been  convincing  to  the  test  subjects,  there 
are  no  clear  treasures  of  operational  effectiveness  which  can 
be  tuantified  or  analyzed.  Given  that  the  computer  capacity 
is  available  to  the  location  of  the  decision  maker,  STAF'MER2 
would  appear  to  he  a  welcome  addition  to  the  repertoire  cf 
decision  aids  available  to  a  commander.  While  this 
addresses  the  original  objective  namely  the 
demonstration  and  evaluation  of  STAMMER2,  it  cannot 
full/  answer  the  question  of  overall  utility  of  the  system 
to  a  decision  maker  in  a  real  military  environment. 

Due  to  the  experimental  nature  of  STAMMER2,  operator 
action  was  required  for  the  message  input  procedure. 
The  requirement  to  leave  the  explanation  trace  before  the 
next  message  can  be  read  is  purely  artificial  and  an 
accepted  penalty  in  order  to  run  the  program  simultaneously 
with  WES.  In  an  operational  system,  the  mechanise  for  the 
transfer  of  data  between  a  sensor  ana  the  decision  maker 
should  be  understood  and  automatic.  The  addition  of 
procedural  steps  such  as  function  key  or  terminal  inputs   in 


c  r  d  e  r  t  c  receive  a  ser.scr  re  pert  may  terd  t<~  "be 
ignored  during  times  of  stress.  In  addition,  this  automatic 
transfer  must  be  regulated  to  a  suitable  tine  period  between 
reports.  Reports  from  multiple  sensors  at  short  intervals 
would  rapidly  overload  an  operator  required  to  viev.  every 
report  . 

STAf^MIR2,  as  an  investigation  into  the  utility  of 
artificial  intelligence  as  applied  to  command  and  control, 
has  progressed  beyond  the  concept  and  technology 
demonstration  phase  of  its  existence.  The  somewhat 
artificial  nature  of  the  message  transfer  procedures  is 
capable  of  approaching  a  "real  world"  condition  in  that 
machine  and  human  readable  formatted  nessa?es  are  in  cemmen 
use  as  in  the  RAINFORM  reporting  system.  The 
production  rules  are  similarly  clcse  tc  conditions  which  may 
in  fact  exist  in  an  operational  environment,  although 
an  agency,  such  as  the  Office  of  the  Chief  of  Naval 
Operations,  operating  through  the  Surface  Warfare 
Manpower  and  Training  Requirements  Division  (OF-93), 
who  could  tas£  the  Fleet  Training  Centers  (Atlantic  and 
Pacific),  could  further  refine  them  tc  reflect  mere 
realistic  decision  conditions. 

A  decision  aid,  even  one  directed  toward  a  unique 
user  <e.g.,  the  Tactical  Action  Officer),  may  need  tc  te 
distributed  throughout  a  command  center.  The  mass  of  data 
presented  may  require  the  decentralization  cf  control  tc  the 
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poirt  where  the  1AC  or  staff  watch  officer  function  ray  te 
rele-atec.  to  relatively  discrete  areas  cf 
responsibility.  While  the  requirement  for  a  pri^e  decision 
maker  exists,  this  role  may  well  be  one  of  comrrani  by 
negation.  There  is  a  tire  penalty  for  reevalua  t  ing 
previously  screened  information  in  a  high  density 
environment.  lach  decision  raker  would  then  require  his  own 
version  of  STAWIP.2  that  contains  a  tailored  ruip  set. 
The  present  implementation  of  STAMMEE2  already  contains 
rules  which  may  be  useful  and  transportable  to  this 
practical  application. 

The  benefit  of  this  program  aboard  a  ship,  cr  at  a  rrajcr 
Fleet  command  center,  for  example,  could  be  that  a  "super 
agency",  consisting  of  intelligence  and  warfare  specialists 
could  provide  a  realistic  assessment  of  situations  which 
right  be  encountered  and  tailor  propositions  to  reflect 
standardized  intelligence,  doctrine,  and  tactics  in  a  pre- 
loaded database  and  system  that  would,  in  real  time,  be  an 
"oracle"  which  wculd  provide  assistance  and  guidance  tc  the 
decision  maker.  In  case  of  questionable  data  the  review 
capability  of  the  process  by  which  a  decision  was  reached 
would  assist  the  decision  maker  in  logically  analyzing  his 
own  thought  process.  The  training  benefit  cf  the 
proposition  review  could  increase  the  competence  cf  the  TAC 
who  could  see  how  the  "panel  of  experts"  approached  the 
problem. 


In   summary,   this   thesis   has   presented   background 
information   concerning    artificial    intelligence   and 

STAMER2.    Various   testing  methods   were   presented    to 

evaluate   the   usefulness   of  5TA.MMER2  and  an  experiment  was 

conducted  and  examined  which  used   ene  of   these  methods. 

STAPM3LE2  appears  to  he  a  useful  decision  aid   concept   with 

great   potential   for   further  research  and  testing. 
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APPENDIX  A 
STAMMER2  GUIDE 

The  following  is  a  procedural  guide  to  beginning 
tne  STAMMERS  program  for  this  experiment.  After  logging  into 
the  Naval  Postgraduate  School  Corrrrand,  Control, 
and  Communications  Laboratory  UNIX  system  (DEC  PDP 
11/70),  a  TELNET  link  is  established  to  the  TOPS20  system 
at  Naval  Ocean  Systems  Center,  San  Liege,  California,  under 
account  tare  NPS2.  Following  the  prompt,  the  entries 
with  asterisks  are  entered.  Carriage  returns  (<cr>)  ere 
only  recuired  as  indicated. 


G  stammers  <cr> 

LOAD (TIMPHAK.COM]  * 

compiled  on  24-Feb-&l  ie:29:29 

PILE  CREATED  24-Feb-Sl  l£:2h:5c 

(DISPLOB  redefined) 

(IN-LANE  redefined) 

(RESULTPRINT2R  redefined, 

TEMPHAKCOMS 

(SENS0RANG5  reset) 

<NPS2>TSMPEAK.C0M.l 

_(SZTC  HOMESEIP  (SZTC  CWNSEIP  'BELKNAP))  * 

(CWNSEI?  reset) 

lEOKISEIP  reset) 

EILKNAP 

_(STAMMER)  * 

Welcome  to  version  2.5  of  the  STAMMER  TSA  system. 

Memory  file?  (Default  is  MEMORY.):  MEM. J?  <cr>    * 

Memory  initialized. 

Rulefile?  (Default  is  RULES .) :RULES.JI  <cr> 

Rules  loaded 

What  file  would  ycu  like  to  take  messages  fror? 

(Default  is  SCENE. ICE; :  SCENE. JF   <cr> 

Are  ycu  running  on  a  Tektronix?no 

Do  ycu  nave  a  Tektronix  available  for  display?  nc  * 
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Passive  detection,  heard  SPS40  at  bearing  Zbb  .  Zy      T  i  rre  t  12 
Associated  with  track  BERKELEY 

RiPOlT:  BERKELEY  was  sighted  in  the  merchant  lane  LANI1 
Question?   Quit 
Leaving  EXPLAIN 

Passive  detection.  Heard  SPS39  at  bearing  Z-tB  .29  Time:  12 
Associated  with  track  BERKELEY 

REPORT:  BERKELEY  was  sighted  in  the  merchant  lane  LAKE1 
Question?   BREAK 
(Explain  broken) 
:  d  r  i  b  b  I  e  ] 
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APPENDIX  B 

DATA  BASE 

This  is  the  data  base  from  which  STAMMER2  will  draw- 
its  first  assertions.  It  has  teen  specifically  tailored  tc 
the  experiment  scenario  ana  dees  not  reflect  real- 
world  conditions.  STAMMER2  will  update  its  data  base  as 
ressaees  are  received. 


iMERCEANTLANE  LANE1) 

(LOCATION  LAN  El  ((53.33  -32.1) (t£. 9 

-27.25))) 

(OWNSHI?  EELKNAP) 

(ID  BELKNAP  FRIEND) 

(ID-AMPLIFY  BELKNAP  ML-EATTLE) 

(ID  BERKELEY  ERIEND) 

v ID-AMPLIFY  BERKELEY  MIL-BATTLE) 

(ID  KNCX  ERIEND) 

(ID-AMPLIiY  KNCX  MIL-BATTLE) 

(ID  U201  HOSTILE) 

(ID-AMPLIFY  U001  MIL-BATTLE) 

(ID  See4  ERIEND) 

(ID-AMPLIFY  S0£4  MIL-AUXIL, 

(ID  S025  FRIEND) 

(ID-AMPLIFY  5025  MIL-AUXIL) 

(ID  MSR1  FRIEND) 

(ID-AMPLIFY  MER1  MIL-AUXIL) 

(ID  MER1  FRIEND) 

(ID-AMPLIFY  MER1  MIL-AUXIL) 

(ID  MER1  FRIEND) 

(ID-AMPLIFY  MER1  MIL-AUXIL) 

(ID  MER2  FRIEND) 

(ID-AMPLIFY  MER2  MIL-AUXIL) 

(ID  MER3  FRIEND) 

( ID-AMPLIFY  MER3  MIL-AUXIL; 

(ID  S22S  HOSTILE) 

(ID-AMPLIFY  Se£&  MIL-BATTLE) 

(ID  S206  HOSTILE) 

(ID-AMPLIFY  S006  MIL-BATTLE) 

(ID  S227  HOSTILE) 

(ID-AMPLIFY  S0£7  MIL-BATTLE) 

(ID  HOST4  HOSTILE) 
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(IP-AMPLIFY  H0ST4  MIL-EATTLE) 

STCF 
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APPENDIX  C 

RULES 

The  following  are  the  ^reduction  rules  upon  which 
STAMMER2  operates.  As  data  from  the  data  base  or  messages 
satisfy  the  conditions  in  each  of  the  rules  (labelled  * 
<LITEM>),  the  data  stream  is  built.  When  sufficient  data 
exists,  the  rules  fire  and  the  appropriate  actions  are 
carried  out.  The  plain  text  PRIN'FCRM  statement  is  the  man- 
readable  explanation  of  the  assertion  which  has  been  built 
in  STAMMER2. 


INHERIT 

(CONDITIONS  ((ALIAS  *PLAT  *UNKNOWN ) 
(TYPE  *PIAT  *TY?) 
(ID  *FLAT  *ID1) 
(IB-AMPLIFY  -PLAT  *IDMP) 
(CLASS  *PLAT  *CLS) 
(MEDIUM  *PLAT  *MED) ) 
ACTIONS 

((TYPE  *UNKNCWN  *TYP ) 
(ID  -UNKNOWN  *ID1) 
(ID- AMPLIFY  ^UNKNOWN  -IDMP) 
(CLASS  *UNKNC'*N  *CLS) 
(MEDIUM  ^UNKNOWN  -MED,') 
CONF  1 .0  PRINFCRM 

'if  an  unknown  is  identified,  it  inherits  the  properties  of 
its  identification.") 

KNGWN-PLAT 

(CONDITION'S  ((SIGHTING  *P  *S  ) 

(^UNLESS*  (GWNSHIP  *P ) ) 
(FULLY-KNOWN  *P) 
(ID- AMPLIFY  *P  *I)) 
ACTIONS 

( (IDENTIFIED  *?)) 
CONF  1 .0  PRINFOHM 
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"r"ark  as  identified  if  known  with  certainty") 

NOT-LAST-SIGHT  IN G-VER2 

(CONDITIONS  ((SIGHTING  *PLAT  *S1) 

(PREDECESSOR  *S1  *S2) 

('-NOT*  (SAME-AS  *S2  NIL)') 

(*UNLESS*  (NOT-LAST  *S2))1 
ACTIONS 

(  (NOT-LAST  *S2)) 
CONE  1.0  PRINFCfiM 

If   a   sighting  has  a^predecessor ,  then  that  predecessor  is 
not  the  last  sighting.") 

NOT-FIRST-SIGHTING 

.CONDITIONS  ((SIGHTING  *PLAT  *S1) 

(*UNLESS*  (NOT-FIRST  *S1)) 
(PREDECESSOR  *S 1  *S2) 
(*NOT*  (SAME-AS  *S2  NIL))) 

ACTIONS 

(  (NOT-FIRST  *S1) ) 

CONF  1  .0  PRINFORM 

"if   an  earlier   sighting   occurs  ,  record  that  the  previous 
sighting  is  not  the  first  sighting.") 

LAST -VIEW 

(CONDITIONS  ((SIGHTING  -PLAT  *S1) 

(^UNLESS*  (NOT-LAST  *S1 ) ) ) 

ACTIONS 

((LAST-SIGHTING  -PLAT  -Si)) 

CONF  .99  PRINFORM 

"If   the   sighting  is  not  followed,  it  is  the  last  sighting. 
(.99;") 

FIRST -VIEW 

(CONDITIONS  ((SIGHTING  -'PLAT  *S1) 

(*UNLESS*  (NOT-FIRST  *Sl))) 

£ C  TION  S 

("(FIRST-SIGHTING  -PLAT  *S1)) 

CONF  .99  PRINFORM 

"if   the  sighting  is  not  preceded,  it  is  the  first  sighting. 
(  .99)") 

SIMPLY-REACHABLE 

(CONDITIONS  ((CONTACT  *CONT) 

(FIRST-SIGHTING  *CCNT  *S 1 ) 
(SIGHTING  *PLaT  *S2) 
(ID-AMPLIFY  -PLAT  MIL-SATTLE) 
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(*NOT*  (SAME-AS  *CONT  -PLAT)) 

(*UNLESS*  (OWNSEIP  *PL£T)) 

(POSITION  *S1  *P1) 

(POSITION  *S2  *P2, 

( TOS  *S1  *T1) 

(TOS  *S2  *T2) 

(SWR  *P1  *T1  *P2  *T2) ) 
ACTIONS 

((SIMPLY-WITHIN-RZACH  *S1  *S2)) 
CONF  .98  PRINFORM 

"if   a   contact's   sighting   could   travel  to  a  MIL-BATTLE's 
sighting,  then  they  are  sirnply  reachable.  (.98)") 

REACHABLE 

(CONDITIONS  ((CONTACT  *CONT) 

(SIGHTING  *C0NT  *S1) 

(SIGHTING  *PLAT  *S2) 

(*NCT*  (SAME-AS  *PLAT  *CONT)) 

(-UNLESS*  (C'A'NSHIP  *PLAT}) 

(SIMPLY-WITHIN-REACE  *S1  *S2) 

(*UNLESS*  (BLCCKED-FROM  *S1  *S2))) 
ACTIONS 

( (WITHIN-REACE  *S1  *S2)) 
CONF  .97  PRINFORM 

If  twc  sightings  are  within  reach  of  each  ether,  and  are 
not  blocked  by  patrol  overflights,  then  are  reachable  from 
each  other.  ( .9?)"  ) 

COULB-BE-COMBATANT 

(CONDITIONS  ((CONTACT  *C0NT) 

(FIRST-SIGHTING  *CONT  *S1) 

(ID-AMPLIFY  -PLAT  MIL-BATTLE) 

(^UNLESS*  (OWNSEIP  -PLAT)) 

(LAST-SIGHTING  *?LAT  -S2) 

(WITEIN-RSACE  *S1  -S2 ) ) 
ACTIONS 

((KNOWN-COMBATANT  *CONT); 
CONF  .15  PRINiORM 

'if   a  contact's   position  could   be  reached   by   a  known 
combatant,  then  the  contact  might  be  a  cenbatant  (.15).") 

NOT-KNOWN-COMBATANT 

(CONDITIONS  ((CONTACT  *CONT) 

(*UNLZSS*  (KNOWN-COMBATANT  *CCNT))) 
ACTIONS 

((TYPE  *C0NT  MERCHANT)) 
CONF  .45  PRINFORM 
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contact   could   not   be   any 


deterrrineo.  by  rule  COULD-BE-CCMBATANT ) ,  t 
merchant  ( .45) . " ) 


know  n 

ben 


FCSS-F.PT 

(CONDITIONS 


combatant 

it   ray 


((PATROL  *PTL) 

(CONTACT  ♦CONT) 

(SIGHTING  *CCNT  -Si) 

(SIGHTING  -PLAT  *S2 ) 

(ID-AMPLIFY  *PLAT  MIL-BATTLE) 

(♦UNLESS*  (OVNSHIP  *PLAT)) 

(SOURCE  *S2  *PTL) 

(*NOT*  (SAME-AS  *S1  *S2)) 

(♦UNLESS*  (DISSIMILAR  *CONT  *PLAT))) 
ACTIONS 

('.POSSIBLE-REPORT  *CONT  *?TL)) 
CONE  .95  PRINFGRM 


If  a  patrol  si^nts  a  MIL-BATTLE  platform,  and  a  contact  is 
similar  to  thet platform,  then  the  patrol  report  concerns  the 
contact.  (.95)") 


BLOCKER 


^CONDITIONS  ( 


CONTACT  *CC,\'T) 

SIGHTING  -CCNT  *Sl) 

SIGETING  *PLAT  *S2) 

ID-AMPLIFY  *PLAT  MIL-BATTLE) 

*NOT*  (SAMS-AS  *CCNT  *PLAT)) 

♦UNLESS*  (OWNSHIP  *PLAT)) 

PATROL  *PTL) 

♦UNLESS*  (POSSIBLE-REPORT  *CONT  *PTL)) 

SIGETING  *FTL  *S3) 

NOT-LAST  *S3) 

SUCCESSOR  *S3  *S4) 

FOSITICN  *S1  *P1) 

POSITION  *S2  *P2) 

POSITION  -S3  *?3) 

POSITION  *S4  *P4, 

TCS  *S1  *T1) 

TOS  *S2  *T2) 

TOS  -S3  *T3; 

TOS  -S4  *T4) 

-OR*  (CROSS PATHS  -PI  *P2  *P3  *P4) 
(GRAZE  *P1  *P2  *F3  *P4) 5 

-NOT*  (WINT-BEFORZ  *P1  *T1  *P2  *T2  *P3  -T3 
*T4;  ) 

*NOT*  (WENT-AETER  -PI  *T1  *P2  *T2  *P3   -T3  -?4 
*T4)  )  ) 
ACTIONS 

( (BLOCKED-FRCM  *S1  *S2; ) 
CCNF  .9  PP.INFCRM 


*P4 
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If  a  path  between  two  sighting  has  aot  teen  detected  by  a 
patrol,  and  it  would  have  if  they  were  sightings  c:  the  same 
vessel,  then  they  are  different  vessels.  (.9)") 

CREA7EDETZCT 

(CONDITIONS  ((SIGHTING  *?LAT  *SGT) 

(*UNLESS*  (IDENTIFIED  -PLAT)) 
(*UNLZSS*  (DETECTION  -PLAT)) 
(SOURCE  *SGT  EW}) 
ACTIONS 

( (DETECTION  -PLAT)) 
CONF  1.0  PRINFORP 

"if  the  source  of  a  sighting  is  EW,  then  mark  it  detected.") 

CREATECONTACT 

(CONDITIONS  ((SIGHTING  *FLAT  *SGT) 

(*UNLZSS*  (IDENTIFIED  *PLAT  )  ) 
(*UNLESS*  v  CONTACT  '-PLAT)) 
(SOURCE  *SGT  RADAR) ) 
ACTIONS 

( ^CONTACT  *PLAT) ) 
CONE  1.0  PRINFCRM 

"if   radar   is  the  source  of  a  platfcrr's  sighting,  then  the 
platform  is  a  contact.') 

CREATEPLAT 

.CONDITIONS  ((SIGHTING  *?LAT  *SGT) 

(*UNLESS*  (CWNSHIF  -PLAT)) 
(*UNLESS*  (PLATFORh  -PLAT))) 
ACTIONS 

( (PLATFORM  *PLAT)) 
CONF  1.0  PRINFCRM 

"Every  sighting  is  a  platform,  except  the  own  ship.  ) 

SMALL-CRAFT9 

(CONDITIONS  ((CONTACT  *WHO) 

(FIRST-SIGHTING  -WHO  *S1) 

(SOURCE  *S1  RADAR) 

(RANGE  *S1  -Rl) 

(LESS-THAN  -Rl  6) 

(STRENGTH  -SI  STRCNG) ) 
ACTIONS 
( (TYPE  *WEO  SUE) 

(hODE  *<aHO  SURFACE)  ) 
CONF  .t   PRINFCRM 

"If   the  range  of  a  strong  radar  sighting  is  less  than  3  r, 
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and  it  is  the  first  sighting,  then  the  contact  is  oossitly  a 
surface  sub .  (  .t )    ) 

SNAIL-CRAFTS 

(CONDITIONS  ( (CONTACT  *X ) 

(SIGHTING  *X  *SIGHT) 
(NOT-FIRST  *SIGHT) 
(RANGE  *SIGE'T*R) 
(LESS-TEAN  *R  16) 
(GREATER-THAN  *P.  9) 
(STRENGTH  *SIGET  WEAK) 
(SPEED  *SIGHT  *SPD) 
(-NOT*  (GREATER-THAN  *S?D  22))) 
ACTIONS 

( (*OR*  (TYPE  *X  FISHING) 
(TYPE  *X  PATROL) 
(TVPE  *X  SUB) )) 
CONE  .15  PHINFORP 

If  the  range  cf  a  weak  sighting  is  between  9  and  15,  and 
the  speed  is  less  than  22,  then  the  contact  is  possibly  e 
sub  cr  a  patrol  or  a  fishing  vessel.  (.15)") 

SMALL-CRAFTS 

(CONDITIONS  ((CONTACT  *WHO) 

(SIGHTING  *WHO  *Sl) 

(NOT-IIRST  *S1) 

(SOURCE  *S1  RADAR) 

(RANGE  *S1  GRANGE) 

(LESS-TEAN  *RANGE  16) 

(GREATER-THAN  -RANGE  9) 

(STRENGTH  *S  1  WEAK) 

(SPEED  *S1  *S?EED) 

(GREATER-THAN  *SPEED  22)) 
ACTIONS 
( (*OR*  'TYPE  *WH0  SUB) 

(TYPE  *¥HO  PATROL) ) ) 
CONF  .3  PRINFCRf" 


"if  the  range  of  a  weak  radar  sighting  is  be 
and  the  speed  is  fcreater  than  t|20,  then  t 
possibly  a  sub  or  a  patrol.  (.3)') 

SivALL-CHi-.ET4: 

(CONDITIONS  ((CONTACT  ^UNKNOWN) 

(SIGHTING  ^UNKNOWN  *SIGETING1) 
(LAND-DIST  *SIGETING1  *DIST) 
(SOURCE  -SIGHTING1  RADAR) 
(RANGE  *SIGETING1  -RANGE) 
(LESS-THAN  -RANGE  9) 
(GREATER-THAN  GRANGE  3) 


etween  9  end  16, 
he   contact   is 
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(STRENGTH   ♦SIGHTI.vGl   Wl 
(LESS-TEAN    -LIST    50) ) 


;AK) 


ACTIONS 
((*OB*  (TYPE 

(TYPE 
(TYPE 
(TYPE 
(TYPE 


-UNKNOWN 
♦UNKNOWN 
•UNKNOWN 
♦UNKNOWN 

-UNKNOWN 


3  US  ) 

SHORE-PATROL) 
PLEASURE s 
COWIRCIAL) 
LANDING)  )  ) 


.1    PR  INFORM 


If  the  range  cf  a  weak  sighting  is  between  3  and  9,  and  the 
distance  from  land  of  the  sighting  is  less  than  50,  then  the 
vessel  ray  be  a  sub,  a  patrol,  a^pleasure  craft,  a  landing 
craft,  cr  a  commercial  craft.  (.1)") 

SMALL— CRAET3 

(CONDITIONS  ((CONTACT  ♦UNKNOWN) 

(SIGHTING  ♦UNKNOWN  ♦SIGHTING) 

(LAND-BIST  ♦SIGHTING  ♦DIST) 

(SOURCE  ♦SIGHTING  RADAR} 

(RANGE  ♦SIGHTING  ♦RANGE) 

(LESS-TEAN  ♦RANGE  9) 

(GREATER-THAN  ♦RANGE  3) 

(STRENGTH  ♦SIGHTING  WEAK) 

(GREATER-THAN  ♦DIST  50)) 
ACTIONS 

( (TYPE  ♦UNKNOWN  SUB) ) 
CON?  .55  PRINECRM 

"if  the  range  of  a  wea*c  radar  sighting  is  between  3  and  9, 
and  the  sighting  is  further^than  50  miles  from  land,  then 
the  contact  is  a  sub.  (.35)') 


SMALL-CRAET2 

(CONDITIONS 


((CONTACT  ♦UNKNOWN) 
(SIGHTING  ♦UNKNOWN  ♦SIGHTING) 
(NOT-EIRST  ♦SIGHTING) 
(SOURCE  ♦SIGHTING  RADAR ) 
(RANGE  ♦SIGHTING  ♦RANGE) 
I LESS-TEAN  ♦RANGE  3) 
(STRENGTH  ♦SIGHTING  WEAK) 
(SPEED  ♦SIGHTING  ♦SPEED) 
(♦NOT#  (GREATER-THAN  ♦SPEED  3 

ACTIONS 

( (♦OR^  (TYPE  ♦UNKNOWN 
(TYPE  ♦UNKNOWN 
(TYPE  ♦UNKNOWN 

CONE  .12  PRINEORh 


)  )  ^ 


DEBRIS  ) 
SUB) 
BUOY) )) 


"if  the  weak  radar  sighting  is  not  known  to  be  moving  faster 
than  3   knots,  then  the  contact  is  either  a  buoy,  a  sub,  or 


7? 


debris.  (.12)") 

SMALL-CRAFTl 

(CONDITIONS  ((CONTACT  *UMNOWN) 

( S I GHT I N  G  *  U  i\  1 N  C  'a  N  *S  I G  FT  I N  C- ) 

(NOT-FIRST  ^SIGHTING) 

(SOURCE  ^SIGHTING  RaDAR  ) 

I  RANGE  -SIGHTING  *RANGE) 

(LESS-THAN  *RANGE  3) 

(STRENGTH  -SIGHTING  WEAK) 

(SPEED  -SIGHTING  *S?EED) 

(GREATER-THAN  -SPIED  3)) 
ACTIONS 
( (TYPE  ^UNKNOWN  SUB; 

(MODE  ♦UNKNOWN  PERISCOPES  CR%  SNORKEL)) 
CONE  .6  PRINFORM 

"if  the  week  radar  sighting  is  moving  at  greater  then  3 
knots,  then  tne  contact  is  a  sub  in  either  periscope  or 
snorkel  mcie .  (.€)") 


ID-IAN E 
(CCNDIT 


IONS 


((SIGHTING  *SHIP  -SIGHTING) 

(M2RCEANTLANE  -LANE) 

(PLATFORM  *SEIP) 

(LOCATION  *LANE  *LANEL0C) 

(POSITION  -SIGHTING  *PCS  ) 

(IN-LANE  -LANELOC  *P0S )  ) 
ACTIONS 
( (INSIEE-A-MERCEANTLANE  ♦SIGETING) 

(♦REPORT*  -SHIP  "  was  sighted  in  the 
CONE  l.e  PRINFORM 


merchant  lane   -LANE' 


"if   a  ship  is  sighted  within  some  merchantlane,  then  record 
that  it  is  inside  that  lane."; 


INSIDS-A-STORM 

(CONDITIONS  ((STORM  ♦STGRM) 

(PLATFORM  -SHI?  ) 

(♦UNLESS*  » IDENTIFIED  -SHIP)) 
(LOCATION  -STORM  ♦STMLOC) 
(SIGHTING  -SHIP  -SIGHTING) 
(POSITION  ^SIGHTING  ♦FOS ) 
(INSIDE  *?OS  *STML0C) ) 

ACTIONS 

((TYPE  *SEIP  MERCHANT) 
(-REPORT*  -SHIP  "  was  sighted  inside 

CCNF  -.25  PRINFORM 


-STORM)) 


"if   a  ship  is  sighted  inside  a 
be  a  merchant.  ( .25) ' ) 


storm,  then  the  ship  may  not 
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CICSE-PO?lT? 

(CONDITIONS  ((CONTACT  -SLIP) 

(FIRST-SIGHTING  *SHIP  ^SIGHTING) 
(RANGE  ^SIGHTING  ♦RANGE) 
(LESS-THAN  GRANGE  12) ) 
ACTIONS 

( (TYPE  ♦SEI?  MERCHANT) ) 
CON?  -.2  PRINFGRM 

If  the  first   sighting  of  a  ship  is  within  12  nm,  then  it 

[ray  not  be  a  merchant.  (.2)") 

DISTANT-POPUP 

(CONDITIONS  ((CONTACT  *SEIP) 

(FIRST-SIGHTING  ♦SHIP  ♦SIGHTING) 
(RANGE  ♦SIGHTING  -RANGE) 
(GREATER-TFAN  '"RANGE  30)) 
ACTIONS 

((TYPE  ♦SHIP  MERCHANT) ) 
CONE  -.2  PRINEORh 

"if   the   range  of  the  first  signting  is  greater  than  30  nm, 
then  the  ship  mignt  not  be  a  merchant.   (.2)") 

COURSE-CHANGED 

(CONDITIONS  (  (CONTACT  ♦SHIP) 

(SIGHTING  ♦SEIF  ♦SIGETINGl) 
(NOT-IIRST  ♦SIGETINGl) 
(NOT-LAST  *SIGHTING1) 
(SUCCESSOR  ^SIGETINGl  ♦S IGFTING2 ) 
(COURSE  *SIGHTING1  ♦COURSEl) 
(COURSE  ♦SIGHTING2  ♦C0URSE2) 

(♦UNLESS*   (ROUGHLY-TEE-SAKE-COUP.SE-AS  ♦COURSE1 

♦COURSES))) 
ACTIONS 

( (TYPE  *SEIP  MERCHANT) ) 
CONE  -.3  PRINFCRM 

'if   the  course  has  changeditsignif icantly ,  then  the  sighting 

ma:/  not  be  a  merchant.  (.3)") 

S  PE^"E  — C  FT  *  N  G  E "D 

.CONDITIONS ( (CONTACT  *SEIP) 

(SIGHTING  ♦SHIP  ♦SIGHTING) 

(NCT-EIRST  ♦SIGHTING) 

1NOT-LAST  -'SIGHTING) 

(SUCCESSOR  ♦SIGHTING  ♦SIGHTING2) 

(SPEED  ♦SIGHTING  ♦SPEEDl) 

(SPEED  ♦SIGETING2  ♦SPEEZ2) 

(♦UNLESS^    (RCUGHLY-THE-SAME-SPEED-AS   ♦SPEEDl 


?9 


"$?llb<c  )  )  ) 
ACTIONS 

( (TYPE  *SEIP  MERCHANT) ) 
CON?  -.3  PRINFCRM 

If   the   speed  has  changed  significantly,  then  the  sighting 
may  not  he  a  merchant.  (.3)") 

FASTER-THAN-A-MIRCHANT 

(conditions  ((contact  *seip) 

(sighting  *ship  -sighting) 
(not-first  ^sighting) 
(speed  -^sighting  *speed) 

(GREaTIR-THAN  *SPEED  25)) 
ACTIONS 

( (TYPE  *SKIF  MERCHANT) ) 
CCNF  -.25  PPINFCRM 


If  the  speed n  is 
merchant .  ( .25)") 


greater  than  25  kncts,  then  it  is  net  a 


SLC'vER-THAN-A-MERCHANT 
'CONDITIONS  ((CONTACT  *SEIF) 

(SIGHTING  *SHIF  -SIGHTING) 
(NOT-FIRST  ^SIGHTING) 
^SPEED  ^SIGHTING  *SPEED) 
(LFSS-TEAN  *SPEED  9)) 
ACTIONS 

( (TYPE  *SEIP  MERCHANT) ) 
CONE  -.15  PRINFORM 


If   tne   speedM  is 
merchant .  ( .15)") 


less  than  9  knots,  then  it  may  net  te  a 


MATCH-PLAT 
■.CONDITIONS 


( (FIRST-SIGHTING  *?LAT1  *SGTl) 
(NOT-LAST  *SGT1) 
(SUCCESSOR  *SGT1  *SGT1S ) 
^LAST-SIGHTING  *PLAT2  *SGT2) 
(*UNLESS*  (OWNSHIP  *PLAT1 ) ) 
(*UNLSSS*  (O'A'NSHIP  *PLAT2)) 
(#NOT*  (SAME-AS  *PLAT1  -PLAT2  )  ) 
(POSITION  *SGT1S  *P0S1S) 
(SPEED  *SGT1S  *SPD1) 
(POSITION  *SGT1  *POSl) 
(COURSEERCM  *POSl  *PCS1S  *CRS1) 
(TOS  *SGT1  *T1) 
(POSITION  *SGT2  *P0S2  ) 
(TOS  *SGT2  *T2) 
(LESS-THAN  *T2  *T1) 
(COURSEFROM  *P0S2  *POSl  *CRS2) 


oZ 


"If  the 

sane,  and 
sighting's 
cf  the  sa 


(SPEEDFROM  *PCS2  *T2  *P0S1  *T1  *SPL2) 
(ROUGHLY-THE-SAME-COURSE-AS  *CRS1  *CRS2) 
(ROUGHLY-TEE-SAME-SPEEE-AS  *SPD1  *SP22)j 

ACTIONS 

((ALIAS  *?IAT2  *PLAT1) ) 

CON?  .b   PRINSORM 

course  and  speed  of  two  sightings  are  roughly  the 
3  if  one  sighting's  position  vould  te  the  ether 
's  extrapolated  position,  then  the  two  siehtings  are 
a  roe  vessel .  (  .5)  ) 


OUTSIEE-ALL-LANES 

(CONDITIONS  ((SIGHTING  *SHIP  ^SIGHTING) 

(-UNLESS-  (IDENTIFIED  *SHIP)) 
v PLATFORM  -SHIP) 
(*UNLESS*  (MEDIUM  *SHIF  AIR)) 

(*UN12SS*  (INSIDE-A-MERCHANTLANE  *SIGETING))) 
ACTIONS 

((TYPE  *SHIP  MERCHANT) ^ 
CONF  -.08  PRINFCRM 

"If   a   sighting   is   outside   all   merchant  lanes,  then  the 
vessel  might  not  be  a  merchant.  (.06)") 

STOP 
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APPENDIX  D 


r^ISSAuES 


This  is  the  message  file  which  STAMMER2  will 
reed,  in  sequential  order  (vice  time  order),  during  the 
execution  of  the  scenario.  It  was  built  by  putting  WES 
generated  reports  into  STAMMERS  readable  forn. 


BELKNAP  64.316  -26.9?  0) 
BERKELEY  RADAR  64.3  -29.265  02) 


KNOX 
rSRl 
MER2 
E021 
E001 
E002 
Z002 
E001 


RADAR 


RADAR 


64.3  -25.615  23) 

64.2  -20.3  0) 
RADAR  64.622  -29.167  0) 
EW  270.0  K57  24) 

EXTERNAL    274.0    BERKELEY    54.3    -29.12 
EW    92.0    K2    12) 


25) 


EXTERNAL  90.0  BERKELEY  64.250  -29.15  14) 
EXTERNAL  267.0  BERKELEY  64.333  -29.133  13) 

EELKNAP  64.4  -26.9?  20) 

KNOX   RADAR    64.367    -25.734    22) 

E201    EXTERNAL    263.0    BERKELEY    64.367    -29.15?    22) 

E002  EXTERNAL  92.2  BERKELEY  64.36?  -29.157  22) 

S204  EXTERNAL  64.716  -29.333  26) 

S004  EXTERNAL  54.580  -29.435  35) 

U201  EXTERNAL  64.350  -26.333  30) 

BELKNAP  64.435  -26.97  40) 

BERKELEY  RADAR  64.4  -29.26?  40) 


KNOX 
S005 
S004 
E003 
E001 
U001 
E003 
S205 
S004 
S005 
S234 
U001 
U001 
E224 
S006 


RADAR    64.433    -25.632    40) 


EXTERNAL 
EXTERNAL 
EW    342.0 

EXTERNAL 
EXTERNAL 
EXTERNAL 
EXTERNAL 
EXTERNAL 
EXTERNAL 
EXTERNAL 
EXTERNAL 
EXTERNAL 
EXTERNAL 
EXTERNAL 


64.433  -32.0  5  40) 

64.667  -29.433  40) 

DONKA  41) 

236.0  BERKELEY  64.433  -29.267  40 

64.316  -26.50  40) 

359.0  EERKELEY  64.467  -29.30  46) 

64.467  -32.216  45 ) 

-29.457  45) 

-29.964  50 ) 

-2  9.50  52) 

-26.646  50) 

-26.646  50) 
-26 
59) 


64.532 
64.465 
64.616 
64.350 
64.350 


47.0  KNOX  64.5 
64.646  -29.333 


(E004  EXTERNAL  47.0  KNOX  64.5 


534 


5?^ 
59; 


62 


(BELKN 

;BEfiKI 

KNOX 

E803 

S02  5 
S004 

S026 
1204 
E003 
US  01 
5  02? 
S225 
S208 
BELKN 
BERKS 
(KNOX 


AP  54.570  -28.97  62) 

LEY  RADAR  64.5  -£9.367  60) 

RADAR  54.5  -28.534  62 ) 

EXTERNAL    205. 2    BERKELEY   64.534   -29.383    6£ 

IXTZENAL  64.534  -29.837  65) 

EXTERNAL    64.57  2    -29.62    65) 

EXTERNAL    64.832    -29.333    65] 

EXTERNAL  4c. 0  KNOX  64.551  -28.485  72) 

EXTERNAL  222.2  BERKELEY  64.57  2  -29.433  75) 

EXTERNAL  64.367  -28.752  75) 

EXTERNAL  64.766  -27.866  74) 

EXTERNAL  64.572  -29.815  75) 

EXTERNAL  64.267  -29.849  75) 

AP  64.532  -28.97  82) 

LEY  RALAR  64.57  -29.333  82) 

RAEAR  64.566  -26.433  62) 


APPENDIX  I 
SCENARIO 

This  is  the  scenario  which  will  be  generated  by  the 
Warfare  Environmental  Simulator  (WES>  program,  resident  in 
the  TENSX  system  at  NOSC,  San  Diego,  California.  This  file 
sets  initial  ship  identity,  position,  course  and  speed, 
and  a  basic  file  of  orders  to  be  carried  cut  by  each  side. 
The  orders,  which  represent  contingency  plans,  are 
executed  immediately  following  the  initialization  of  WARGAN 
and  the  entry  of  PLAYER  which  is  the  user/interactive 
program  for  WIS. 


NORTH 
YES 
YES 
SHI? 

1.1  BERKL  EERK 
N00001  AaAA  922 
64-l£N   29-e5'w'   33?   15 
SHIP 

1 .2  KNOX  KNOX 
N00002  EBEB  921 

64-1  EN   ££-52 W   £32   15 
SHIP 

1.3  3ELKN   BZLKN 
N00003  CCCC  902 

64-1 9N   28-55 V  222      15 
SHIP 

7.1  VOROS   ?RES2 
Nlllll  EEEE  904 
55-06N   29-32W   245   1? 
SHIP 

7.2  SKORY   KASE 
N11112  FEFE  9?5 
65-12N   29-20'*   150   22 
SHIP 
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7.3  BURL   KCTLI 
N11113  GGGG  926 
65-05N   27-22'a"   217   16 
SHIP 

7.4  VAZNY   DEGPI 
N11114  BEHH  3  27 
64-22N   32-25';.'   142   17 
SHIP 

7.5  SSGN9   CEARI 
N1111E  IIII  926 
64-20N   27-52W   275   25 
SHIP 

7.6  ^ERl   KAZBK 
N22221  K*KK  322 
c4-l£N   30-16*   222   16 
SHIP 

7.7  KER2   KAZBK 
N22222  LLLL  321 
64-52N   29-12W   215   16 
SHIP 

7.6  MER3   KAZBK 

N'22223  MMMM  922 

54-16N      32-22W      236      17 

0  R  u  £  R  S 

BLUE  PLAN  ALFA 

FOR  BELKN  REPORT  ALL  ALL  TIME  1  999 

FOR  EERKL  REPORT  ALL  ALL  TIME  1  999 

FOR  KNOX  REPORT  ALL  ALL  TIME  1  999 

PLACE  A  MARKER  64-12N  30-45W  TIME  1  999 

PLACE  A  MARKER  64-36N  29-30W  TIME  1  999 

PLACE  A  MARKED  65-18N  28-20W  TIME  1  999 

C RANGE  PLAN  ALFA 

FOR  SKORY  REPORT  ALL  SURFACE  TIME  1  999 

FOR  VAZNY  COURSE  245  TIME  45 

FOR  BURL  SPEED  22  TIME  55 

FOR  VOROS  REPORT  ENEMY  AIR  TIME  3   94 

FOR  EURL  REPORT  ALL  ALL  TIMI  22  922 

FOR  VAZNY  REPORT  ALL  SURFACE  TIME  1  999 

FOR  SSGN9  REPORT  ALL  SURFACE  TIME  12  24 

FOR  MER1  REPORT  ALL  SURFACE  TIME  1  999 


by: 
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